Opening a Queue in TagCollector

Login to TagCollector web-portal and click on Create New Release link. Then follow as guided below:

  • Set a name for the new release cycle for CMSSW such as 3_10_X.
  • Set Build-Phase to closed in order to prevent automatic queueing of tags in the release cycle.
  • Set Build-Type to Development.
  • Set Build-Status to Started in order to make the release_name visible in the TagCollector interface.
  • Set the required architecture to slc5_ia32_gcc434 as currently it is the only production architecture available.

Updating Scripts for corresponding IB

  • Checkout the existing collection of IB scripts (and QA test scripts) from SVN repository to a local build machine (e.g. lxbuildXXX, lxbuildXXX etc.)
       - svn -q co https://svn.cern.ch/reps/CMSIntBld/branches/prod/IntBuild/IB
       - svn -q co https://svn.cern.ch/reps/CMSIntBld/branches/prod/IntBuild/QA
       - svn -q co https://svn.cern.ch/reps/CMSIntBld/branches/prod/IntBuild/web
       

Updating Configuration for Building an IB
  • Edit IB/config.py (Add IB build configuration section for the new release cycle. ConfigTag contains build rules collection while ToolBoxTag contains external tool collection.)
  • Save and quit from the file
  • For cross-checking, issue the command svn diff; it should only report the above-mentioned changes usually.
  • Now update the configuration in the SVN by issuing the command svn ci -m "a comment describing about the updated configuration" IB/config.py
    For reference, consider a sample section to be added in the config.py file for the new release cycle,
    #####################################################################################################
    # configuration for 3.10 cycle
    
    rel = '3.10'
    relCycles.append(rel)
    Configuration[rel] = {}
    Configuration[rel]['releaseTag'] = 'CMSSW_3_10_X'
    Configuration[rel]['tagCollTag'] = 'CMSSW_3_10_X'
    Configuration[rel]['scramVers']  = 'V2_2_0'
    Configuration[rel]['configTag']  = { 'slc5_ia32_gcc434'  : 'V03-30-03',
                                         'slc5_amd64_gcc434' : 'V03-30-03', }
    Configuration[rel]['toolBoxTag'] = { 'slc5_ia32_gcc434'  : '17.0-cms11',
                                         'slc5_amd64_gcc434' : '17.0-cms11', }
    ##################################################################################################### 
       

Create an AFS volume for the IgProf QA Test results
Ask an AFS space admin (MarttiP, AndreasP, StephenG, GillesR) to create two new volumes for the IB like this:
afs_admin create -q 20G /afs/cern.ch/cms/sdt/web/qa/igprof/data/slc5_ia32_gcc434/3.11 q.cms.sdtqaigp11311
afs_admin create -q 20G /afs/cern.ch/cms/sdt/web/qa/igprof-testbed/data/slc5_ia32_gcc434/3.11 q.cms.sdtqaigp12311
   
(replace the architecture string and the release cycle numbers accordingly).

Updating Configuration for Running QA Tests
  • Edit QA/configQA.py (Add a section for current release cycle. We need to know the hardware resources to be specified for the new release cycle after having groupwide consensus internally. If any previous section has to be removed due to its deprecated phase etc. we will have to update the crontab appropriately.)
  • Now, issue the command svn diff; it should only report the above-mentioned changes usually.
  • Now update the configuration in the SVN by issuing the command svn ci -m "a comment describing about the updated configuration" QA/configQA.py
    For reference, consider a sample section to be added in the configQA.py file for the new release cycle,
    #####################################################################################################
       siteInfoQA = { 'cern.ch'  : { 'testBoxes' :
                                    {'3.10' : ['lxbuildXXY','lxbuildXXX','lxbuildXXX'],
                                     '3.9' : ['lxbuildXXX', 'lxbuildXXX','lxbuildXXX'],
                                   },
                                 },
                   'testing'  : { 'testBoxes' :
                                  {'3.6' : ['lxbuildXXX'],
                                   },
                                }
                          }
    ##################################################################################################### 
       

Updating Web scripts area to include new relese cycle
  • Edit web/cgi-bin/showIB.py to add entry for the new release cycle
  • Check for any accidentally introduced syntactical errors during the intervention of python script with the command python -c 'import showIB'
  • Now update the configuration in the SVN by issuing the command svn ci -m "a comment describing about the updated configuration" web/cgi-bin/showIB.py
    For reference, consider an excerpt from the constructor in the showIB.py where the new release cycle is supposed to be added.
    #####################################################################################################
     self.cycList = ['3.10', '3.9', '3.8']
    ##################################################################################################### 
       

Updating cronjobs for the new Release Cycle

For this, one has to work under cmsbuild account (i.e. ssh cmsbuild@.cern.ch)

  • Issue the command acrontab -e and make a section for the new release cycle (We need to know the hardware resources to be specified for the new release cycle after having groupwide consensus internally. In case of getting/introducing new build machines, one has to keep in mind to aslo configure the cleanup scripts for those machines.) The new section needs to include the following scripts:
     - newIB.sh
     - testWatcher.sh (for removing stale entries if any....)
     - stdTests.sh (executes QA tests)
     - cleanupLocalStdTetDirs.py (for cleaning up stuff generated by *stdTests.sh*)
     - cleanupLocalBuildDisk.py (This script should be run for cleaning old IBs stuff on the machines having build process executed on them.)

All the above-mentioned tests (scripts) reside in <cmsbuild's home>/scripts/ directory and are carried out on our production architecture only i.e. slc5_ia32_gcc434.

Consider a sample section below:

* To build the IB

######################################################################################################################################
# for the 3.10 on slc5_ia32_gcc434
# sleep a bit longer to avoid conflicts in the TC when creating the tag
10 02,13 * * * vocmsXX (sleep 5 && /afs/cern.ch/user/c/cmsbuild/scripts/newIB.sh 3.10 slc5_ia32_gcc434 )
00,15,30,45 * * * * vocmsXX /afs/cern.ch/user/c/cmsbuild/scripts/testWatcher.sh
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#
# for the 3.10 on slc5_amd64_gcc434
# sleep a bit longer to avoid conflicts in the TC when creating the tag
15 02,13 * * * vocmsXX (sleep 5 && /afs/cern.ch/user/c/cmsbuild/scripts/newIB.sh 3.10 slc5_amd64_gcc434 )
00,15,30,45 * * * * vocmsXX /afs/cern.ch/user/c/cmsbuild/scripts/testWatcher.sh
#####################################################################################################################################
      

* To Cleanup the local build area

####################################################################################################################################
# for 3.10 cycle 
17 1 * * * vocmsXX /afs/cern.ch/user/c/cmsbuild/scripts/cleanupLocalBuildDisk.py
17 1 * * * vocmsXX /afs/cern.ch/user/c/cmsbuild/scripts/cleanupLocalBuildDisk.py
####################################################################################################################################
      

* To start the QA for a release cycle

####################################################################################################################################
# QA tests for different release cycles (example is for 3.10)
20 * * * * lxbuildXXX /afs/cern.ch/user/c/cmsbuild/scripts/stdTests.sh 3.10 slc5_ia32_gcc434
####################################################################################################################################
      

* To cleanup the QA local test

####################################################################################################################################
# cleanup stdTest/QA area on the machines where we run QA tests
30 22 * * * lxbuildXXX /afs/cern.ch/user/c/cmsbuild/scripts/cleanupLocalStdTestDirs.py --topDir /build/cmsbuild
30 22 * * * lxbuildXXX /afs/cern.ch/user/c/cmsbuild/scripts/cleanupLocalStdTestDirs.py --topDir /build/cmsbuild
30 22 * * * lxbuildXXX /afs/cern.ch/user/c/cmsbuild/scripts/cleanupLocalStdTestDirs.py --topDir /build/cmsbuild
####################################################################################################################################
      

-- AdeelRehman - 14-Oct-2010

Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r8 - 2010-12-09 - AndreasPfeiffer
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback