GIT

Adam LCS git repository can be found under: https://git.cern.ch/reps/adam

The git repository uses the following directory structure:

• external - all external projects built from sources are stored in this directory
• {subproject name} - every subproject has its own directory named with the subbproject name
• mock-cfgs - all mock configurations (e.g. architecture specific, build specific) are stored in this directory

The master brunch is used for the main development activity. The staging branch is used for problem fixing in the current release candidate.

In order to make a release an annotated tag in the staging branch has to be created.

Build system

The Build system is CMake based for in-house projects. For external projects that are being built from sources the original solution used for building is preserved. Each subproject contains at least 4 directories:

• cmake - for cmake modules
• doc - for documentation
• packaging - for RPM packaging specific things
• src - for the source code

The packaging directory contains:

• rpm directory
  • spec directory - contains respective spec files
  • Makefile - a short script that starts the build

In order to build an RPM package:

1. go to rpm directory, e.g.: cd demo/packaging/rpm
2. run: make rpm

In order to build with mock:

1. create a mock directory in packaging folder, e.g.: mkdir demo/packaging/mock
2. copy respective mock configuration to the mock directory, e.g.: cp mock-cfgs/epel-6-adam-ci-x86_64.cfg demo/packaging/mock/build.cfg
3. go to rpm directory, e.g.: cd demo/packaging/rpm
4. run: make mock

Mock configurations:

• the configuration name starts with the name of targeted platform, e.g. epel-6
• than comes the adam interfix
• afterwords is the build type, either: ci (continuous integration), nightly (nightly build), staging (release candidate), production (production release)
• finally the configuration name ends with the targeted architecture, e.g.: x86_64

CMake:

• the cmake build system should provide by default 4 targets: default (for building the subproject), doc (for generating documentation), test (for running unit tests) and install (for installing binaries)

Spec file implements the process of generating RPM packages, amongs others:

• in the build section (%build):

   
       mkdir build # create build directory
       cd build # enter build directory
       cmake28 -D CMAKE_BUILD_TYPE=RelWithDebInfo -D CMAKE_INSTALL_PREFIX='' .. # run cmake with RelWithDebInfo flag (this ensures that debug-info packages will be created)
       make %{?_smp_mflags} # the default target builds the binaries
       make doc # the doc target generates the documentation (goes to the devel package)
     

• in the check section (%check):

      cd build # enter build directory
      make test # the test targe runs unit tests
     

• in the install section (%install):

      cd build # enter build directory
      make install DESTDIR=%{buildroot} # the install target installs the binaries
     

YUM

The YUM repository is a AFS based website, it is hosted at: /afs/cern.ch/project/adam/www/repo and can be accessed through: http://cern.ch/adam-repo

The website contains 4 YUM repositories:

• ci - contain RPM packages generated during continuous integration process
• nightly - contain RPM packages generated during nightly builds
• staging - release candidate RPM packages
• production - contain the production RPM packages

In order to make the folder accessible to the web server read access had been granted to webserver:afs user:

fs setacl /afs/cern.ch/project/adam/www/repo webserver:afs rl

Also, in order to allow jenkins user to upload RPM packages and to run createrepo write access had been granted to cern:nodes user, e.g.:

fs setacl /afs/cern.ch/project/adam/www/repo/ci/slc6/i386 cern:nodes rlidwk

For details on how to configure access rights for an AFS folder see:

 http://information-technology.web.cern.ch/services/fe/afs/howto/configure-afs-access-rights 

Also, for details on how to administer an AFS project (e.g. how to set quotas) see:

 http://information-technology.web.cern.ch/book/afs-administrator-guide 

Access control for the website is managed through a .htaccess file, details in how to configure user authentication and authorization can be found under:

 https://espace2013.cern.ch/webservices-help/websitemanagement/ConfiguringAFSSites/Pages/AccesscontrolonAFSsites.aspx 

Jenkins

Adam Jenkins instance for the LCS system can be accessed at https://jenkins.cern.ch/adam

In case of any problems the CERN IT department should be contacted (currently the responsible person is: terje.andersen@cern.ch)

A Continuous Integration Jenkins project uses following options:

• Discard Old Builds - controls disk consumption
  • Days to keep builds: 5
  • Max # of builds to keep: 10

• Source Code Management:
  • the git repository is accessed using a service user: adamsrv
  • every Jenkins project needs to checkout the sources of respective subproject e.g. demo directory and the directory containing mock configurations: mock-cfgs
  • clean before checkout: cleans up the workspace before every checkout by deleting all untracked files and directories; including those which are specified in .gitignore, also resets all tracked files to their versioned state; ensures that the workspace is in the same state as if you cloned and checked out in a brand-new empty directory, and ensures that your build is not affected by the files generated by the previous build
  • Polling ignores commits in certain paths: Jenkins will pay attention only to included regions / will pay no attension to excluded regions when determining if a build needs to be triggered
  • Sparse Checkout paths: list of path that should be checkout (this way only necessary things are checkout and not the whole repository)

• Build Triggers
  • Build after other projects are built: specifies the list of upstream projects
  • Poll SCM: specifies how often Jenkins should query git repository for changes, uses cron job format, e.g.:

     H/30 * * * * 

    triggers a query every 30 minutes

• Build
  • for building and archiving the artefacts a shell script is used, e.g.:

            mkdir demo/packaging/mock # create mock directory
            cp mock-cfgs/epel-6-adam-ci-x86_64.cfg demo/packaging/mock/build.cfg # copy respective mock configuration
            cd demo/packaging/rpm/ # enter rpm directory
            make mock # start the build using mock tool
            cp mockbuild/*.el6.x86_64.rpm /afs/cern.ch/project/adam/www/repo/ci/slc6/x86_64 # copy RPM packages into respective directory on AFS
            createrepo /afs/cern.ch/project/adam/www/repo/ci/slc6/x86_64 # run createrepo inorder to create / update the YUM repository
            cp rpmbuild/SRPMS/* /afs/cern.ch/project/adam/www/repo/ci/slc6/SRPMS # copy source RPMs into respective directory on AFS
            createrepo /afs/cern.ch/project/adam/www/repo/ci/slc6/SRPMS # run createrepo inorder to create / update the YUM repository
            cd - # go back to the main root directory
            rm -rf * # clean the workspace
           

  • the mock tool is being used because it automatically installs the dependencies (and doesn't require root access in order to do so) and ensures that build time dependencies are correct

A nightly-build Jenkins project uses following options:

• Discard Old Builds - controls disk consumption
  • Days to keep builds: 5
  • Max # of builds to keep: 10

• Source Code Management:
  • the git repository is accessed using a service user: adamsrv
  • nightly-build project needs to checkout the whole git repository (https://git.cern.ch/reps/adam)
  • clean before checkout: cleans up the workspace before every checkout by deleting all untracked files and directories; including those which are specified in .gitignore, also resets all tracked files to their versioned state; ensures that the workspace is in the same state as if you cloned and checked out in a brand-new empty directory, and ensures that your build is not affected by the files generated by the previous build

• Build Triggers
  • Build periodically: specifies how often Jenkins should execute the nightly build, e.g.:

     2 0 * * * 

    triggers the build every day at 2:00

• Build
  • for building and archiving the artifacts a shell script is used (similarly as in case of a Continuous Integration Jenkins project)