Difference: SiteConfInGitlab (1 vs. 30)

Revision 302019-09-13 - ChristianPovedaRuiz

Line: 1 to 1
 
META TOPICPARENT name="FacilitiesServices"
Line: 51 to 51
 

Instructions to add a project/site

Added:
>
>
Run the createSiteConfProject.py script. If you want to add the project manually, follow these instructions:
  Go to - https://gitlab.cern.ch/projects/new:
    • Project owner: Groups-->siteconf

Revision 292019-09-12 - ChristianPovedaRuiz

Line: 1 to 1
 
META TOPICPARENT name="FacilitiesServices"
Line: 19 to 19
  The SITECONF GitLab is setup as a group (SITECONF) with a project inside this group for each site (CMS sitename, i.e. TN_CO_Name). This is done to allow direct write access for site administrators to their and only their SITECONF section. Permissions of SITECONF are:
  • owner of the GitLab group is cms-comp-ops-site-support-team e-group. This allows the site support team to create new/remove inactive sites and create/modify files to assist sites.
Changed:
<
<
  • developer of each GitLab project are site administrators as specified in SiteDB. (The site admin list is updated from SiteDB once a day.) This allows site admins to create/modify files for their site (= update the master branch).
>
>
  • developer of each GitLab project are the site administrators and executives in the corresponding cms-<facility>-admin and cms-<facility>-exec e-groups. For more information check the SITECONF permissions managment section.
 
  • visibility of the GitLab projects is restricted to cms-authorized-users e-group. This allows anyone in CMS to check out the files of a site and "propose" a change (= create a merge request).

Revision 282019-09-11 - ChristianPovedaRuiz

Line: 1 to 1
 
META TOPICPARENT name="FacilitiesServices"
Line: 90 to 90
  Go to - https://gitlab.cern.ch/SITECONF/<cms_site_name>/edit, scroll down and press button "Remove project"
Added:
>
>

SITECONF permissions managment

The access to each of the site's SITECONF project is done via LDAP using the facility mailing lists for executives and administrators in CERN e-groups. This means that users in the cms-<facility>-admin and cms-<facility>-exec can modify the https://gitlab.cern.ch/SITECONF/<site>/ project. It is not recommended to give users access manually using gitlab.

When a new site is created, the updateSiteConfMembers.py will fetch the new site's information from CRIC, create a new gitlab group for the members of each of e-group mailing lists, and give those groups access to the SITECONF project. This script runs automatically every 8 hours. In order for this to work, the site must be registered in CRIC, the SITECONF project must exist and the mailing lists must have the corresponding facility name in CERN e-groups.

 

META TOPICMOVED by="rmaciula" date="1461074454" from="CMSPublic.SiteConfMigrationToGitlab" to="CMSPublic.SiteConfInGitlab"

Revision 272019-09-10 - StephanLammel

Line: 1 to 1
 
META TOPICPARENT name="FacilitiesServices"
Line: 8 to 8
  The JobConfig part of SITECONF needs to be accessible to jobs running at the site. CMS sites have two options for this:
  1. they can maintain a local copy of the SITECONF section of their site on the worker nodes (or a network filesystem mounted on the worker nodes). The SITECONF section needs to be located at ${CMS_PATH}/SITECONF/local/JobConfig/... Most installations use ${CMS_PATH}/SITECONF/<sitename>/... and make ${CMS_PATH}/SITECONF/local a link to it.
Changed:
<
<
  1. they can use the CERN Virtual Machine FileSystem, CVMFS, of CMS. The CMS CVMFS provides an up-to-date copy of the SITECONF areas of all sites. Site administrators configure the CVMFS mount at their site such that /cvmfs/cms.cern.ch/SITECONF/local points to their site entry. This is done by setting CMS_LOCAL_SITE in /etc/cvmfs/default.local. The CMS CVMFS support team (Bockjoo Kim) keeps the CVMFS copy synchronized with the repository via a cron job that runs every few hours.
>
>
  1. they can use the CERN Virtual Machine FileSystem, CVMFS, of CMS. The CMS CVMFS provides an up-to-date copy of the SITECONF areas of all sites. Site administrators configure the CVMFS mount at their site such that /cvmfs/cms.cern.ch/SITECONF/local points to their site entry. This is done by setting CMS_LOCAL_SITE in /etc/cvmfs/default.local
    <!--(or using the CVMFS default of CMS_LOCAL which points to /etc/cvmfs/SITECONF/local)-->
    . The CMS CVMFS support team (Bockjoo Kim) keeps the CVMFS copy synchronized with the repository via a cron job that runs every few hours.
 

GitLab Organization

Revision 262019-05-16 - StephanLammel

Line: 1 to 1
 
META TOPICPARENT name="FacilitiesServices"
Line: 80 to 80
 
      • Multiple recipients can be specified, separated by whitespace. More info
      • uncheck "Jira Jira issue tracker".
  • Add CMSSST user as Master to a new project:
Changed:
<
<
>
>
 
    • Select members to invite: CMSSST
    • Choose a role permission: Master
    • Press button "Add to project"

Revision 252018-07-23 - XiaoweiJiang

Line: 1 to 1
 
META TOPICPARENT name="FacilitiesServices"
Line: 80 to 80
 
      • Multiple recipients can be specified, separated by whitespace. More info
      • uncheck "Jira Jira issue tracker".
  • Add CMSSST user as Master to a new project:
Changed:
<
<
>
>
 
    • Select members to invite: CMSSST
    • Choose a role permission: Master
    • Press button "Add to project"

Revision 242018-05-02 - RokasMaciulaitis

Line: 1 to 1
 
META TOPICPARENT name="FacilitiesServices"
Line: 79 to 79
 
      • [Save changes]
      • Multiple recipients can be specified, separated by whitespace. More info
      • uncheck "Jira Jira issue tracker".
Added:
>
>
 

Instructions to delete a project/site

Revision 232018-01-30 - StephanLammel

Line: 1 to 1
 
META TOPICPARENT name="FacilitiesServices"
Line: 70 to 70
 
    • git add JobConfig/site-local-config.xml
    • git commit -a
    • git push -u origin master
Changed:
<
<
>
>
  • select "protected branches" in the config pulldown hidden in the upper right corner of the web page (https://gitlab.cern.ch/SITECONF/<SITE_NAME>/protected_branches) (current GiLab release "Settings" on the left menu bar, then "Repository", expand "Protected Branches", choose Branch "master", Allowed to merge "Developers+Masters", Allowed to push "Developers+Masters" push "Protect"):
 
    • check the "Developers can push" checkbox for the master branch
Changed:
<
<
  • select "services" in the config pulldown hidden in the upper right corner of the web page:
>
>
  • select "services" in the config pulldown hidden in the upper right corner of the web page (current GitLab release "Settings" on the left menu bar, then "Integration"):
 
Changed:
<
<
      • Multiple recipients can be specified, separated by whitespace. More info.
>
>
      • Multiple recipients can be specified, separated by whitespace. More info
      • uncheck "Jira Jira issue tracker".
 

Instructions to delete a project/site

Revision 212017-01-11 - RokasMaciulaitis

Line: 1 to 1
 
META TOPICPARENT name="FacilitiesServices"
Line: 62 to 62
 
Deleted:
<
<
 
  • insert at least an empty JobConfig/site-local-config.xml file into the repository asap (Empty repositories have erroneous archives!)
    • git clone ssh://git@gitlab.cern.ch:7999/SITECONF/<sitename>.git
    • cd <sitename>
Line: 78 to 70
 
    • git add JobConfig/site-local-config.xml
    • git commit -a
    • git push -u origin master
Added:
>
>
 

Instructions to delete a project/site

Revision 202016-11-27 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="FacilitiesServices"
Added:
>
>
 

SITECONF Repository

SITECONF is a repository for site configuration files. These configuration files contain CMS site information required for jobs running at the site (Trivial File Catalogue, TFC; DBfrontier/squid location; address of the data popularity service) and PhEDEx configuration information. The first set of information is kept under "JobConfig" in the "site-local-config.xml" file. The information in SITECONF is maintained by the site administrators.
SITECONF is not the only source of site configuration information in CMS. The repository is structured with an area, the sitename, for each site, i.e. SITECONF/<sitename>/JobConfig/site-local-config.xml.

Revision 192016-07-22 - StephanLammel

Line: 1 to 1
Changed:
<
<
META TOPICPARENT name="TWiki.WebPreferences"
>
>
META TOPICPARENT name="FacilitiesServices"
 

SITECONF Repository

Revision 182016-07-12 - StephanLammel

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"

SITECONF Repository

Line: 51 to 51
 

Instructions to add a project/site

Changed:
<
<
Go to - https://gitlab.cern.ch/projects/new.
  • Project path: https://gitlab.cern.ch/SITECONF/cms_site_name
  • Decription: cms_site_name site configuration
  • double check Visibility Level is "Private" (Visibility is set at the group level to xxx for members of the egroup yyyy so CMS members have all read access.)
  • Under <project> --> Settings --> Rename repository
    • change the path from tx_co_name to the propper upper/lower case string
    • click "Rename project"
>
>
Go to - https://gitlab.cern.ch/projects/new:
    • Project owner: Groups-->siteconf
    • Project name: <sitename>
    • Import project from: <leave as is>
    • Description: <sitename> site configuration
    • Visibility Level: Private (Visibility is set at the group level to xxx for members of the egroup yyyy so CMS members have all read access.)
    • [Create Project]
  • select "edit project" in the config pulldown hidden in the upper right corner of the web page:
  • select "protected branches" in the config pulldown hidden in the upper right corner of the web page (https://gitlab.cern.ch/SITECONF/<SITE_NAME>/protected_branches):
    • check the "Developers can push" checkbox for the master branch
  • select "services" in the config pulldown hidden in the upper right corner of the web page:
 
  • insert at least an empty JobConfig/site-local-config.xml file into the repository asap (Empty repositories have erroneous archives!)
Changed:
<
<
  • Allow pushing to protected branches for developers. Go to - https://gitlab.cern.ch/SITECONF/<SITE_NAME>/protected_branches and check "Developers can push" field
  • GitLab provides an option to send e-mail notifications whenever commits are pushed to a repository. From a project's page, navigate to "Settings", then "Services" and open "Emails on Push" and check the "Active" box. Multiple recipients can be specified, separated by whitespace. More info.
>
>
    • git clone ssh://git@gitlab.cern.ch:7999/SITECONF/<sitename>.git
    • cd <sitename>
    • mkdir JobConfig
    • touch JobConfig/site-local-config.xml
    • git add JobConfig/site-local-config.xml
    • git commit -a
    • git push -u origin master
 

Instructions to delete a project/site

Revision 172016-06-20 - RokasMaciulaitis

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"

SITECONF Repository

Line: 52 to 52
 

Instructions to add a project/site

Go to - https://gitlab.cern.ch/projects/new.

Changed:
<
<
>
>
 
  • Decription: cms_site_name site configuration
  • double check Visibility Level is "Private" (Visibility is set at the group level to xxx for members of the egroup yyyy so CMS members have all read access.)
  • Under <project> --> Settings --> Rename repository
Line: 65 to 65
 

Instructions to delete a project/site

Changed:
<
<
Go to - https://gitlab.cern.ch/siteconf/<cms_site_name>/edit, scroll down and press button "Remove project"
>
>
Go to - https://gitlab.cern.ch/SITECONF/<cms_site_name>/edit, scroll down and press button "Remove project"
 

Revision 162016-06-06 - StephanLammel

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"

SITECONF Repository

Line: 32 to 32
 
Added:
>
>
Changes to the site admin or site executive list in SiteDB as well as registrations with GitLab take about 12 hours to propagate, i.e. for the account to get access priviledges!
 After your SSH key is registered you can check out the config files of your site, (= project), update some files, and apply the changes:
  1. clone the project of your site
    • $ git clone ssh://git@gitlab.cern.ch:7999/SITECONF/<name_of_site>.git

Revision 152016-06-03 - GastonLyonsPacini

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"

SITECONF Repository

Line: 57 to 57
 
  • Under <project> --> Settings --> Rename repository
    • change the path from tx_co_name to the propper upper/lower case string
    • click "Rename project"
Added:
>
>
  • insert at least an empty JobConfig/site-local-config.xml file into the repository asap (Empty repositories have erroneous archives!)
 
  • Allow pushing to protected branches for developers. Go to - https://gitlab.cern.ch/SITECONF/<SITE_NAME>/protected_branches and check "Developers can push" field
  • GitLab provides an option to send e-mail notifications whenever commits are pushed to a repository. From a project's page, navigate to "Settings", then "Services" and open "Emails on Push" and check the "Active" box. Multiple recipients can be specified, separated by whitespace. More info.
Deleted:
<
<
  • insert at least an empty JobConfig/site-local-config.xml file into the repository asap (Empty repositories have erroneous archives!)
 

Instructions to delete a project/site

Revision 142016-05-20 - StephanLammel

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
Changed:
<
<
<noautolink>
>
>
 

SITECONF Repository

SITECONF is a repository for site configuration files. These configuration files contain CMS site information required for jobs running at the site (Trivial File Catalogue, TFC; DBfrontier/squid location; address of the data popularity service) and PhEDEx configuration information. The first set of information is kept under "JobConfig" in the "site-local-config.xml" file. The information in SITECONF is maintained by the site administrators.
SITECONF is not the only source of site configuration information in CMS. The repository is structured with an area, the sitename, for each site, i.e. SITECONF/<sitename>/JobConfig/site-local-config.xml.

Line: 54 to 57
 
  • Under <project> --> Settings --> Rename repository
    • change the path from tx_co_name to the propper upper/lower case string
    • click "Rename project"
Deleted:
<
<
  • insert at least an empty JobConfig/site-local-config.xml file into the repository asap (Empty repositories have erroneous archives!)
 
  • Allow pushing to protected branches for developers. Go to - https://gitlab.cern.ch/SITECONF/<SITE_NAME>/protected_branches and check "Developers can push" field
  • GitLab provides an option to send e-mail notifications whenever commits are pushed to a repository. From a project's page, navigate to "Settings", then "Services" and open "Emails on Push" and check the "Active" box. Multiple recipients can be specified, separated by whitespace. More info.
Added:
>
>
  • insert at least an empty JobConfig/site-local-config.xml file into the repository asap (Empty repositories have erroneous archives!)
 

Instructions to delete a project/site

Go to - https://gitlab.cern.ch/siteconf/<cms_site_name>/edit, scroll down and press button "Remove project"

Changed:
<
<

>
>
 
META TOPICMOVED by="rmaciula" date="1461074454" from="CMSPublic.SiteConfMigrationToGitlab" to="CMSPublic.SiteConfInGitlab"

Revision 132016-05-20 - RokasMaciulaitis

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
Changed:
<
<
>
>
<noautolink>
 

SITECONF Repository

SITECONF is a repository for site configuration files. These configuration files contain CMS site information required for jobs running at the site (Trivial File Catalogue, TFC; DBfrontier/squid location; address of the data popularity service) and PhEDEx configuration information. The first set of information is kept under "JobConfig" in the "site-local-config.xml" file. The information in SITECONF is maintained by the site administrators.
SITECONF is not the only source of site configuration information in CMS. The repository is structured with an area, the sitename, for each site, i.e. SITECONF/<sitename>/JobConfig/site-local-config.xml.

Line: 56 to 55
 
    • change the path from tx_co_name to the propper upper/lower case string
    • click "Rename project"
  • insert at least an empty JobConfig/site-local-config.xml file into the repository asap (Empty repositories have erroneous archives!)
Changed:
<
<
>
>
  • Allow pushing to protected branches for developers. Go to - https://gitlab.cern.ch/SITECONF/<SITE_NAME>/protected_branches and check "Developers can push" field
  • GitLab provides an option to send e-mail notifications whenever commits are pushed to a repository. From a project's page, navigate to "Settings", then "Services" and open "Emails on Push" and check the "Active" box. Multiple recipients can be specified, separated by whitespace. More info.
 

Instructions to delete a project/site

Go to - https://gitlab.cern.ch/siteconf/<cms_site_name>/edit, scroll down and press button "Remove project"

Changed:
<
<

Old repository structre (git.cern.ch)

  • Managers of the system: @cms-http-group (RW and manage everything)
  • Masters of the system: @cms-comp-ops-site-support-team (RW everything)
  • Guests of the system: @cms-authorized-user (R access to everything)
  • Admins of individual sites: individual users but all of them are members of @cms-authorized-user (acess to write only in his own site's folder)

New structure (gitlab.cern.ch)

Each site admin was able to write only in his own folder. This feature is not available in Gitlab. So now siteconf is not a repository anymore, it is a group and each site has his own project in siteConf group.

  • The owner of a group Siteconf - cms-comp-ops-site-support-team
  • Projects visibility level - Internal. Internal projects can be cloned by any logged in user. They will also be listed on the public access directory (/public) for logged in users. Any logged in user will have Guest permissions on the repository.
  • Site admin - master role in his site project. Site admins are taken from siteDB and synchronized with gitlab once a day.

Instructions

If you already got an announcment about migration, you just need to know that https://git.cern.ch/web/siteconf.git/tree repository is synchronized with https://gitlab.cern.ch/groups/siteconf (check if it is up to date with your local repo) and follow steps below.

  1. You need to login to https://gitlab.cern.ch/. Check if you have a SSH key - https://gitlab.cern.ch/profile/keys. If not, you have to generate it - https://gitlab.cern.ch/help/ssh/README .
  2. Clone project of your site - $ git clone ssh://git@gitlab.cern.ch:7999/siteconf/<name_of_site>.git
  3. Try to make some test changes and do a commit. For example:
    1. $ touch test
    2. $ git add test
    3. git commit -m "Test commit"
    4. git push
  4. Check project of your site if commit was successful - https://gitlab.cern.ch/siteconf/<site_name&gt;
  5. Reset to the previous commit or create new commit without test file (If you did test commit):
    1. $ git reset HEAD^ --hard
    2. Go to https://gitlab.cern.ch/siteconf/<site_name>/protected_branches and unprotect master branch.
    3. $ git push -f
    4. Go again to https://gitlab.cern.ch/siteconf/<site_name>/protected_branches and now set protection for the master branch

Email notifications

If you want to be informed about new commits - https://cern.service-now.com/service-portal/article.do?n=KB0003567

>
>

 
META TOPICMOVED by="rmaciula" date="1461074454" from="CMSPublic.SiteConfMigrationToGitlab" to="CMSPublic.SiteConfInGitlab"

Revision 122016-05-19 - StephanLammel

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"

SITECONF Repository

Line: 54 to 55
 
  • Under <project> --> Settings --> Rename repository
    • change the path from tx_co_name to the propper upper/lower case string
    • click "Rename project"
Added:
>
>
  • insert at least an empty JobConfig/site-local-config.xml file into the repository asap (Empty repositories have erroneous archives!)
 

Instructions to delete a project/site

Go to - https://gitlab.cern.ch/siteconf/<cms_site_name>/edit, scroll down and press button "Remove project"

Revision 112016-05-11 - RokasMaciulaitis

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"

SITECONF Repository

Revision 102016-05-03 - StephanLammel

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"

SITECONF Repository

Line: 50 to 50
 Go to - https://gitlab.cern.ch/projects/new.
Changed:
<
<
  • Visibility Level: Internal (need to update)
>
>
  • double check Visibility Level is "Private" (Visibility is set at the group level to xxx for members of the egroup yyyy so CMS members have all read access.)
  • Under <project> --> Settings --> Rename repository
    • change the path from tx_co_name to the propper upper/lower case string
    • click "Rename project"
 

Instructions to delete a project/site

Go to - https://gitlab.cern.ch/siteconf/<cms_site_name>/edit, scroll down and press button "Remove project"

Revision 92016-05-02 - StephanLammel

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
Changed:
<
<
<noautolink>
>
>
 

SITECONF Repository

SITECONF is a repository for site configuration files. These configuration files contain CMS site information required for jobs running at the site (Trivial File Catalogue, TFC; DBfrontier/squid location; address of the data popularity service) and PhEDEx configuration information. The first set of information is kept under "JobConfig" in the "site-local-config.xml" file. The information in SITECONF is maintained by the site administrators.
SITECONF is not the only source of site configuration information in CMS. The repository is structured with an area, the sitename, for each site, i.e. SITECONF/<sitename>/JobConfig/site-local-config.xml.

Line: 84 to 84
 
    1. Go again to https://gitlab.cern.ch/siteconf/<site_name>/protected_branches and now set protection for the master branch

Email notifications

Changed:
<
<
If you want to be informed about new commits - https://cern.service-now.com/service-portal/article.do?n=KB0003567 </noautolink>
>
>
If you want to be informed about new commits - https://cern.service-now.com/service-portal/article.do?n=KB0003567
 
META TOPICMOVED by="rmaciula" date="1461074454" from="CMSPublic.SiteConfMigrationToGitlab" to="CMSPublic.SiteConfInGitlab"

Revision 82016-05-02 - RokasMaciulaitis

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
Changed:
<
<
>
>
<noautolink>
 

SITECONF Repository

Changed:
<
<
SITECONF is a repository for site configuration files. These configuration files contain CMS site information required for jobs running at the site (Trivial File Catalogue, TFC; DBfrontier/squid location; address of the data popularity service) and PhEDEx configuration information. The first set of information is kept under "JobConfig" in the "site-local-config.xml" file. The information in SITECONF is maintained by the site administrators.
SITECONF is not the only source of site configuration information in CMS. The repository is structured with an area, the sitename, for each site, i.e. SITECONF/<sitename>/JobConfig/site-local-config.xml.
>
>
SITECONF is a repository for site configuration files. These configuration files contain CMS site information required for jobs running at the site (Trivial File Catalogue, TFC; DBfrontier/squid location; address of the data popularity service) and PhEDEx configuration information. The first set of information is kept under "JobConfig" in the "site-local-config.xml" file. The information in SITECONF is maintained by the site administrators.
SITECONF is not the only source of site configuration information in CMS. The repository is structured with an area, the sitename, for each site, i.e. SITECONF/<sitename>/JobConfig/site-local-config.xml.
  The JobConfig part of SITECONF needs to be accessible to jobs running at the site. CMS sites have two options for this:
Changed:
<
<
  1. they can maintain a local copy of the SITECONF section of their site on the worker nodes (or a network filesystem mounted on the worker nodes). The SITECONF section needs to be located at ${CMS_PATH}/SITECONF/local/JobConfig/... Most installations use ${CMS_PATH}/SITECONF/<sitename>/... and make ${CMS_PATH}/SITECONF/local a link to it.
  2. they can use the CERN Virtual Machine FileSystem, CVMFS, of CMS. The CMS CVMFS provides an up-to-date copy of the SITECONF areas of all sites. Site administrators configure the CVMFS mount at their site such that /cvmfs/cms.cern.ch/SITECONF/local points to their site entry. This is done by setting CMS_LOCAL_SITE in /etc/cvmfs/default.local. The CMS CVMFS support team (Bockjoo Kim) keeps the CVMFS copy synchronized with the repository via a cron job that runs every few hours.
>
>
  1. they can maintain a local copy of the SITECONF section of their site on the worker nodes (or a network filesystem mounted on the worker nodes). The SITECONF section needs to be located at ${CMS_PATH}/SITECONF/local/JobConfig/... Most installations use ${CMS_PATH}/SITECONF/<sitename>/... and make ${CMS_PATH}/SITECONF/local a link to it.
  2. they can use the CERN Virtual Machine FileSystem, CVMFS, of CMS. The CMS CVMFS provides an up-to-date copy of the SITECONF areas of all sites. Site administrators configure the CVMFS mount at their site such that /cvmfs/cms.cern.ch/SITECONF/local points to their site entry. This is done by setting CMS_LOCAL_SITE in /etc/cvmfs/default.local. The CMS CVMFS support team (Bockjoo Kim) keeps the CVMFS copy synchronized with the repository via a cron job that runs every few hours.
 

GitLab Organization

Added:
>
>
 The SITECONF repository is currently maintained in CERN's GitLab. GitLab is an enhanced Git-based repository service. The repository is web accessible to all CMS members:
Line: 15 to 14
 The SITECONF repository is currently maintained in CERN's GitLab. GitLab is an enhanced Git-based repository service. The repository is web accessible to all CMS members:
Changed:
<
<
>
>
 The SITECONF GitLab is setup as a group (SITECONF) with a project inside this group for each site (CMS sitename, i.e. TN_CO_Name). This is done to allow direct write access for site administrators to their and only their SITECONF section. Permissions of SITECONF are:
  • owner of the GitLab group is cms-comp-ops-site-support-team e-group. This allows the site support team to create new/remove inactive sites and create/modify files to assist sites.
  • developer of each GitLab project are site administrators as specified in SiteDB. (The site admin list is updated from SiteDB once a day.) This allows site admins to create/modify files for their site (= update the master branch).
Line: 21 to 20
 
  • developer of each GitLab project are site administrators as specified in SiteDB. (The site admin list is updated from SiteDB once a day.) This allows site admins to create/modify files for their site (= update the master branch).
  • visibility of the GitLab projects is restricted to cms-authorized-users e-group. This allows anyone in CMS to check out the files of a site and "propose" a change (= create a merge request).
Deleted:
<
<
 

Instructions for site admins

Changed:
<
<
GitLab provides access via SSH key. To get started, please to the CERN GitLab web page https://gitlab.cern.ch/ and use the standard CERN single sign on, SSO, mechanism to logon.
>
>
GitLab provides access via SSH key. To get started, please to the CERN GitLab web page https://gitlab.cern.ch/ and use the standard CERN single sign on, SSO, mechanism to logon.
 
  • click on "Profile Settings" in the toolbar on the left
  • this changes the toolbar and provides an "SSH Keys" entry, click it
    • add your SSH key if you have one
      • your SSH key is normally stored in ~/.ssh/id_rsa.pub. If you have such a file, take the key from there via
      • cat ~/.ssh/id_rsa.pub
    • otherwise, create an SSH key with the ssh-keygen command and add it
Changed:
<
<
>
>
 After your SSH key is registered you can check out the config files of your site, (= project), update some files, and apply the changes:
  1. clone the project of your site
    • $ git clone ssh://git@gitlab.cern.ch:7999/SITECONF/<name_of_site>.git
Line: 46 to 45
 
  • for a full list click on "Commits" in the tolbar on the left
To discard a previous commit click on the identifier of the commit, 8-digit hex number. This brings you to a page with details about the update. In the upper right there is a button "Revert" that undoes the update in the commit. Please be carefull and check your config files are as desired afterwards!
Deleted:
<
<
 

Instructions to add a project/site

Changed:
<
<
>
>
Go to - https://gitlab.cern.ch/projects/new.
 

Instructions to delete a project/site

Changed:
<
<
>
>
Go to - https://gitlab.cern.ch/siteconf/<cms_site_name>/edit, scroll down and press button "Remove project"
 

Old repository structre (git.cern.ch)

  • Managers of the system: @cms-http-group (RW and manage everything)
  • Masters of the system: @cms-comp-ops-site-support-team (RW everything)
Line: 83 to 84
 
    1. Go again to https://gitlab.cern.ch/siteconf/<site_name>/protected_branches and now set protection for the master branch

Email notifications

Changed:
<
<
If you want to be informed about new commits - https://cern.service-now.com/service-portal/article.do?n=KB0003567
>
>
If you want to be informed about new commits - https://cern.service-now.com/service-portal/article.do?n=KB0003567 </noautolink>
 
META TOPICMOVED by="rmaciula" date="1461074454" from="CMSPublic.SiteConfMigrationToGitlab" to="CMSPublic.SiteConfInGitlab"

Revision 72016-04-27 - StephanLammel

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
Changed:
<
<

SITECONF

>
>

SITECONF Repository

  SITECONF is a repository for site configuration files. These configuration files contain CMS site information required for jobs running at the site (Trivial File Catalogue, TFC; DBfrontier/squid location; address of the data popularity service) and PhEDEx configuration information. The first set of information is kept under "JobConfig" in the "site-local-config.xml" file. The information in SITECONF is maintained by the site administrators.
SITECONF is not the only source of site configuration information in CMS. The repository is structured with an area, the sitename, for each site, i.e. SITECONF/<sitename>/JobConfig/site-local-config.xml.

Revision 62016-04-27 - StephanLammel

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"

SITECONF

SITECONF is a repository for site configuration files. These configuration files contain CMS site information required for jobs running at the site (Trivial File Catalogue, TFC; DBfrontier/squid location; address of the data popularity service) and PhEDEx configuration information. The first set of information is kept under "JobConfig" in the "site-local-config.xml" file. The information in SITECONF is maintained by the site administrators.

Changed:
<
<
SITECONF is not the only source of site configuration information in CMS. The repository is structured with an entry, the sitename, for each site, i.e. SITECONF/<sitename>/JobConfig/site-local-config.xml.
>
>
SITECONF is not the only source of site configuration information in CMS. The repository is structured with an area, the sitename, for each site, i.e. SITECONF/<sitename>/JobConfig/site-local-config.xml.
  The JobConfig part of SITECONF needs to be accessible to jobs running at the site. CMS sites have two options for this:
  1. they can maintain a local copy of the SITECONF section of their site on the worker nodes (or a network filesystem mounted on the worker nodes). The SITECONF section needs to be located at ${CMS_PATH}/SITECONF/local/JobConfig/... Most installations use ${CMS_PATH}/SITECONF/<sitename>/... and make ${CMS_PATH}/SITECONF/local a link to it.
Line: 12 to 12
 

GitLab Organization

Changed:
<
<
The SITECONF repository is currently maintained in CERN's GitLab. GitLab is an enhanced Git-based repository service. The repository is read accessible to all registered CMS members.
>
>
The SITECONF repository is currently maintained in CERN's GitLab. GitLab is an enhanced Git-based repository service. The repository is web accessible to all CMS members:
 
Added:
>
>
 
Changed:
<
<
The SITECONF GitLab is setup as a group (SITECONF) with a project inside thsi group for each site (CMS sitename, i.e. TN_CO_Name). This is done to allow direct write access for site administrators to their and only their SITECONF section. Permissions are set:
  • owner of the GitLab group is cms-comp-ops-site-support-team
  • master role of each GitLab project are site administrators as specified in SiteDB (The site admin list is updated from SiteDB once a day.)
  • visibility of the GitLab projects is restricted to cms-authorized-users e-group
>
>
The SITECONF GitLab is setup as a group (SITECONF) with a project inside this group for each site (CMS sitename, i.e. TN_CO_Name). This is done to allow direct write access for site administrators to their and only their SITECONF section. Permissions of SITECONF are:
  • owner of the GitLab group is cms-comp-ops-site-support-team e-group. This allows the site support team to create new/remove inactive sites and create/modify files to assist sites.
  • developer of each GitLab project are site administrators as specified in SiteDB. (The site admin list is updated from SiteDB once a day.) This allows site admins to create/modify files for their site (= update the master branch).
  • visibility of the GitLab projects is restricted to cms-authorized-users e-group. This allows anyone in CMS to check out the files of a site and "propose" a change (= create a merge request).

Instructions for site admins

GitLab provides access via SSH key. To get started, please to the CERN GitLab web page https://gitlab.cern.ch/ and use the standard CERN single sign on, SSO, mechanism to logon.
  • click on "Profile Settings" in the toolbar on the left
  • this changes the toolbar and provides an "SSH Keys" entry, click it
    • add your SSH key if you have one
      • your SSH key is normally stored in ~/.ssh/id_rsa.pub. If you have such a file, take the key from there via
      • cat ~/.ssh/id_rsa.pub
    • otherwise, create an SSH key with the ssh-keygen command and add it
    • detailed SSH key information is available at https://gitlab.cern.ch/help/ssh/README
After your SSH key is registered you can check out the config files of your site, (= project), update some files, and apply the changes:
  1. clone the project of your site
    • $ git clone ssh://git@gitlab.cern.ch:7999/SITECONF/<name_of_site>.git
  2. update files using your prefered editor, for instance
    • $ vi <name_of_site>/JobConfig/site-local-config.xml
  3. remember that any new files need to be added explicitly, i.e.
    • $ git add <name_of_site>/JobConfig/a_new_file.txt
  4. commit the change, i.e.
    • $ git commit -m "<comment describing the update>"
  5. and push the update to the master branch of SITECONF
    • $ git push
To check the update made it to the SITECONF repository, go to the project in GitLab, i.e. https://gitlab.cern.ch/siteconf/<site_name&gt;, or go to the SITECONF GitLab, click on "Projects" in the main window, and select your project. The project page lists the last commits.
  • for a full list click on "Commits" in the tolbar on the left
To discard a previous commit click on the identifier of the commit, 8-digit hex number. This brings you to a page with details about the update. In the upper right there is a button "Revert" that undoes the update in the commit. Please be carefull and check your config files are as desired afterwards!

Instructions to add a project/site

Instructions to delete a project/site

 

Old repository structre (git.cern.ch)

Revision 52016-04-26 - StephanLammel

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
Changed:
<
<

Siteconf in gitlab.cern.ch

>
>

SITECONF

SITECONF is a repository for site configuration files. These configuration files contain CMS site information required for jobs running at the site (Trivial File Catalogue, TFC; DBfrontier/squid location; address of the data popularity service) and PhEDEx configuration information. The first set of information is kept under "JobConfig" in the "site-local-config.xml" file. The information in SITECONF is maintained by the site administrators.
SITECONF is not the only source of site configuration information in CMS. The repository is structured with an entry, the sitename, for each site, i.e. SITECONF/<sitename>/JobConfig/site-local-config.xml.

The JobConfig part of SITECONF needs to be accessible to jobs running at the site. CMS sites have two options for this:

  1. they can maintain a local copy of the SITECONF section of their site on the worker nodes (or a network filesystem mounted on the worker nodes). The SITECONF section needs to be located at ${CMS_PATH}/SITECONF/local/JobConfig/... Most installations use ${CMS_PATH}/SITECONF/<sitename>/... and make ${CMS_PATH}/SITECONF/local a link to it.
  2. they can use the CERN Virtual Machine FileSystem, CVMFS, of CMS. The CMS CVMFS provides an up-to-date copy of the SITECONF areas of all sites. Site administrators configure the CVMFS mount at their site such that /cvmfs/cms.cern.ch/SITECONF/local points to their site entry. This is done by setting CMS_LOCAL_SITE in /etc/cvmfs/default.local. The CMS CVMFS support team (Bockjoo Kim) keeps the CVMFS copy synchronized with the repository via a cron job that runs every few hours.

GitLab Organization

The SITECONF repository is currently maintained in CERN's GitLab. GitLab is an enhanced Git-based repository service. The repository is read accessible to all registered CMS members. The SITECONF GitLab is setup as a group (SITECONF) with a project inside thsi group for each site (CMS sitename, i.e. TN_CO_Name). This is done to allow direct write access for site administrators to their and only their SITECONF section. Permissions are set:
  • owner of the GitLab group is cms-comp-ops-site-support-team
  • master role of each GitLab project are site administrators as specified in SiteDB (The site admin list is updated from SiteDB once a day.)
  • visibility of the GitLab projects is restricted to cms-authorized-users e-group
 
Deleted:
<
<
Siteconf is a repository to store configuration of CMS Site. These configurations are use by network readonly filesystem - CVMFS. CERN IT decided to migrate from git.cern.ch to gitlab.cern.ch (more info). This page explains the important changes concerning the SITECONF migration from git.cern.ch to gitlab.cern.ch and provides instructions on how to work with it in gitlab.cern.ch.
 
Deleted:
<
<
For questions or report of problems, ask the cms-comp-ops-site-support-team(no-spam)cern.ch .
 

Old repository structre (git.cern.ch)

  • Managers of the system: @cms-http-group (RW and manage everything)
  • Masters of the system: @cms-comp-ops-site-support-team (RW everything)
Line: 36 to 52
 

Email notifications

If you want to be informed about new commits - https://cern.service-now.com/service-portal/article.do?n=KB0003567

Added:
>
>
 
META TOPICMOVED by="rmaciula" date="1461074454" from="CMSPublic.SiteConfMigrationToGitlab" to="CMSPublic.SiteConfInGitlab"

Revision 42016-04-21 - RokasMaciulaitis

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"

Siteconf in gitlab.cern.ch

Changed:
<
<
Siteconf is a repository to store configuration of CMS Site. CERN IT decided to migrate from git.cern.ch to gitlab.cern.ch (more info). This page explains the important changes concerning the SITECONF migration from git.cern.ch to gitlab.cern.ch and provides instructions on how to work with it in gitlab.cern.ch.
>
>
Siteconf is a repository to store configuration of CMS Site. These configurations are use by network readonly filesystem - CVMFS. CERN IT decided to migrate from git.cern.ch to gitlab.cern.ch (more info). This page explains the important changes concerning the SITECONF migration from git.cern.ch to gitlab.cern.ch and provides instructions on how to work with it in gitlab.cern.ch.
  For questions or report of problems, ask the cms-comp-ops-site-support-team(no-spam)cern.ch .

Old repository structre (git.cern.ch)

Line: 18 to 18
 
  • Projects visibility level - Internal. Internal projects can be cloned by any logged in user. They will also be listed on the public access directory (/public) for logged in users. Any logged in user will have Guest permissions on the repository.
  • Site admin - master role in his site project. Site admins are taken from siteDB and synchronized with gitlab once a day.

Instructions

Deleted:
<
<

Migration

  If you already got an announcment about migration, you just need to know that https://git.cern.ch/web/siteconf.git/tree repository is synchronized with https://gitlab.cern.ch/groups/siteconf (check if it is up to date with your local repo) and follow steps below.
  1. You need to login to https://gitlab.cern.ch/. Check if you have a SSH key - https://gitlab.cern.ch/profile/keys. If not, you have to generate it - https://gitlab.cern.ch/help/ssh/README .
Line: 36 to 35
 
    1. Go again to https://gitlab.cern.ch/siteconf/<site_name>/protected_branches and now set protection for the master branch

Email notifications

Changed:
<
<
If you want to be informed about changes in project (commits and etc.) you can specify notification level per group or per project. By default, all projects and groups will use the notification level set above. These setting you can find in your pofile settings. You can find more info - http://doc.gitlab.com/ee/workflow/notifications.html
>
>
If you want to be informed about new commits - https://cern.service-now.com/service-portal/article.do?n=KB0003567
 
META TOPICMOVED by="rmaciula" date="1461074454" from="CMSPublic.SiteConfMigrationToGitlab" to="CMSPublic.SiteConfInGitlab"

Revision 32016-04-19 - RokasMaciulaitis

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
Changed:
<
<

Siteconf migration to Gitlab NOT FINISHED

>
>

Siteconf in gitlab.cern.ch

 
Changed:
<
<
CERN IT decided to migrate from git.cern.ch to gitlab.cern.ch (more info).

Old repository structre

>
>
Siteconf is a repository to store configuration of CMS Site. CERN IT decided to migrate from git.cern.ch to gitlab.cern.ch (more info). This page explains the important changes concerning the SITECONF migration from git.cern.ch to gitlab.cern.ch and provides instructions on how to work with it in gitlab.cern.ch.

For questions or report of problems, ask the cms-comp-ops-site-support-team(no-spam)cern.ch .

Old repository structre (git.cern.ch)

 
  • Managers of the system: @cms-http-group (RW and manage everything)
  • Masters of the system: @cms-comp-ops-site-support-team (RW everything)
  • Guests of the system: @cms-authorized-user (R access to everything)
Changed:
<
<
  • Admins of individual sites: individual users but all of them are members of @cms-authorized-user

New structure

>
>
  • Admins of individual sites: individual users but all of them are members of @cms-authorized-user (acess to write only in his own site's folder)

New structure (gitlab.cern.ch)

Each site admin was able to write only in his own folder. This feature is not available in Gitlab. So now siteconf is not a repository anymore, it is a group and each site has his own project in siteConf group.

 
Changed:
<
<
Each site admin was able to write only in his own folder. This feature is not available in Gitlab. So now siteconf is not anymore repository, it is group and each site has his own project in siteConf group. The owner of a group: cms-comp-ops-site-support-team LDAP Group. Each project has Visibility Level - Internal. Internal projects can be cloned by any logged in user. They will also be listed on the public access directory (/public) for logged in users. Any logged in user will have Guest permissions on the repository. Site admin has master role in his site project. They are taken from siteDB. Site admins from siteDB are updated once a day.

Info for site admis

>
>
  • The owner of a group Siteconf - cms-comp-ops-site-support-team
  • Projects visibility level - Internal. Internal projects can be cloned by any logged in user. They will also be listed on the public access directory (/public) for logged in users. Any logged in user will have Guest permissions on the repository.
  • Site admin - master role in his site project. Site admins are taken from siteDB and synchronized with gitlab once a day.

Instructions

Migration

  If you already got an announcment about migration, you just need to know that https://git.cern.ch/web/siteconf.git/tree repository is synchronized with https://gitlab.cern.ch/groups/siteconf (check if it is up to date with your local repo) and follow steps below.
  1. You need to login to https://gitlab.cern.ch/. Check if you have a SSH key - https://gitlab.cern.ch/profile/keys. If not, you have to generate it - https://gitlab.cern.ch/help/ssh/README .
Line: 27 to 34
 
    1. Go to https://gitlab.cern.ch/siteconf/<site_name>/protected_branches and unprotect master branch.
    2. $ git push -f
    3. Go again to https://gitlab.cern.ch/siteconf/<site_name>/protected_branches and now set protection for the master branch
Added:
>
>

Email notifications

If you want to be informed about changes in project (commits and etc.) you can specify notification level per group or per project. By default, all projects and groups will use the notification level set above. These setting you can find in your pofile settings. You can find more info - http://doc.gitlab.com/ee/workflow/notifications.html

META TOPICMOVED by="rmaciula" date="1461074454" from="CMSPublic.SiteConfMigrationToGitlab" to="CMSPublic.SiteConfInGitlab"

Revision 22016-04-18 - RokasMaciulaitis

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"

Siteconf migration to Gitlab NOT FINISHED

Line: 10 to 10
 
  • Admins of individual sites: individual users but all of them are members of @cms-authorized-user

New structure

Changed:
<
<
Each site admin was able to write only in his own folder. This feature is not available in Gitlab. So now siteconf is not anymore repository, it is group and each site has his own project in siteConf group.
  • Owner: @cms-comp-ops-site-support-team (RW and manage everything)
  • Guests: @cms-authorized-user liked as Developer (R access to everything)
  • Admins of individual sites: master role in their own project

Steps for site admis

>
>
Each site admin was able to write only in his own folder. This feature is not available in Gitlab. So now siteconf is not anymore repository, it is group and each site has his own project in siteConf group. The owner of a group: cms-comp-ops-site-support-team LDAP Group. Each project has Visibility Level - Internal. Internal projects can be cloned by any logged in user. They will also be listed on the public access directory (/public) for logged in users. Any logged in user will have Guest permissions on the repository. Site admin has master role in his site project. They are taken from siteDB. Site admins from siteDB are updated once a day.

Info for site admis

 
Changed:
<
<
When site admin gets notified about migration, he has to do the following steps:
  • update the remote URL to point to the new project in GitLab
  • ...
>
>
If you already got an announcment about migration, you just need to know that https://git.cern.ch/web/siteconf.git/tree repository is synchronized with https://gitlab.cern.ch/groups/siteconf (check if it is up to date with your local repo) and follow steps below.
  1. You need to login to https://gitlab.cern.ch/. Check if you have a SSH key - https://gitlab.cern.ch/profile/keys. If not, you have to generate it - https://gitlab.cern.ch/help/ssh/README .
  2. Clone project of your site - $ git clone ssh://git@gitlab.cern.ch:7999/siteconf/<name_of_site>.git
  3. Try to make some test changes and do a commit. For example:
    1. $ touch test
    2. $ git add test
    3. git commit -m "Test commit"
    4. git push
  4. Check project of your site if commit was successful - https://gitlab.cern.ch/siteconf/<site_name&gt;
  5. Reset to the previous commit or create new commit without test file (If you did test commit):
    1. $ git reset HEAD^ --hard
    2. Go to https://gitlab.cern.ch/siteconf/<site_name>/protected_branches and unprotect master branch.
    3. $ git push -f
    4. Go again to https://gitlab.cern.ch/siteconf/<site_name>/protected_branches and now set protection for the master branch

Revision 12016-03-29 - RokasMaciulaitis

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="TWiki.WebPreferences"

Siteconf migration to Gitlab NOT FINISHED

CERN IT decided to migrate from git.cern.ch to gitlab.cern.ch (more info).

Old repository structre

  • Managers of the system: @cms-http-group (RW and manage everything)
  • Masters of the system: @cms-comp-ops-site-support-team (RW everything)
  • Guests of the system: @cms-authorized-user (R access to everything)
  • Admins of individual sites: individual users but all of them are members of @cms-authorized-user

New structure

Each site admin was able to write only in his own folder. This feature is not available in Gitlab. So now siteconf is not anymore repository, it is group and each site has his own project in siteConf group.

  • Owner: @cms-comp-ops-site-support-team (RW and manage everything)
  • Guests: @cms-authorized-user liked as Developer (R access to everything)
  • Admins of individual sites: master role in their own project

Steps for site admis

When site admin gets notified about migration, he has to do the following steps:

  • update the remote URL to point to the new project in GitLab
  • ...
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback