Using Subversion (svn) in LHCb

The LHCb code is kept in a Subversion repository (currently being migrated from CVS). This page is intended for LHCb Subversion users. Instructions for LHCb Subversion librarians are available on SubversionSupport. Also look at Marco's svn tutorial and the FAQ.

For instructions about the usage of the CVS repository, look at CVSUsageGuidelines.

Quick-Reference for cvs users

CVS2SVNQuickReference

Table of Contents

Accessing Subversion

Web access

• The LHCb, Gaudi and Dirac repositories are world readable and browsable via the two web interfaces and a statistics page
  • LHCb:
    • Trac: https://svnweb.cern.ch/trac/lhcb/browser
    • WebSVN: http://svnweb.cern.ch/world/wsvn/lhcb
    • Statistics: https://svnweb.cern.ch/stats/lhcb/
  • Gaudi:
    • Trac: https://svnweb.cern.ch/trac/gaudi/browser
    • WebSVN: http://svnweb.cern.ch/world/wsvn/gaudi
    • Statistics: https://svnweb.cern.ch/stats/gaudi
  • Dirac:
    • Trac: https://svnweb.cern.ch/trac/dirac/browser
    • WebSVN: http://svnweb.cern.ch/world/wsvn/dirac
    • Statistics: https://svnweb.cern.ch/stats/dirac

The layout of the repository is based on the rules outlined in the Gaudi twiki.

Installing the Subversion client

The Subversion command line client svn is available in all Linux distributions with names like subversion or subversion-client. It is available by default on lxplus5 nodes, but it might not be installed on other machines, but you can easily find on the web how to install it for your distribution.

For Windows, MacOS (and for Linux distributions that do not have it packaged) you can find the sources and pre-compiled distributions linked from the official Subversion web site.

Several graphic clients to Subversion are available too. RapidSVN is a half decent free application and available on many platforms.

On the LHCb Windows Terminal Server we install the CollabNet command line client and the TortoiseSVN graphic client (a command line client is mandatory to be able to use getpack).

Setting up read access

Detailed instructions about how to configure the access to the Subversion repository are provided by IT in the SVN How-To. You can find instructions for Linux and Windows.

Note: To enable the ssh tunnel for the command line client on Windows (URLs of the form svn+ssh://), you may need to add the line

ssh = $SVN_RSH "C:/Program Files/TortoiseSVN/bin/TortoisePlink.exe"

to the [tunnels] section of the subversion client configuration file (either the user or the global one), see the Subversion book for details.

Getting write access

To get write access to LHCb Subversion repository, go to the e-group lhcb-svn-writers and register the account you want to use when accessing the Subversion repository. A mail will be sent automatically to the librarians that will add you to the group.

Note that when you register to the e-group lhcb-svn-writers, you will receive the emails sent on the lhcb-core-soft mailing list.

Connecting with a different user name

For windows, it is quite simple. You just have to change the autologin name in the "svn.cern.ch" session. You can do it by loading the session in putty and change the username in the Connection->Data section as described in the CERN SVN How-To.

For Linux/MacOSX, this is a little bit different because on these platform the username is always forwarded to the ssh connection. In order to change the username that ssh is using, you have to modify or create the ssh configuration file ~/.ssh/config. In that file, the entry for svn.cern.ch should look like:

Host svn.cern.ch svn
   User myusername
   Protocol 2
   ForwardX11 no
   IdentityFile ~/.ssh/id_rsa
Host isscvs.cern.ch
   User myusername    
   Protocol 2
   ForwardX11 no
   IdentityFile ~/.ssh/id_rsa

where "myusername" is the name you want to use for the ssh authentication. Note that the entry for isscvs.cern.ch is needed since the latest getpack also queries this server.

To be noted that this procedure for the setting of the user name prevents the explicit embedding of the user name in the URL of the repository. There is a problem in the release areas and distribution kits if a given user name is hard-coded in the URL. There should be no explicit user name.

Kerberos authentication (Linux/MacOSX)

It is possible to use the Kerberos V token to connect to the Subversion server instead of the ssh key, simply modifying the content of the .ssh/config file as

Host svn.cern.ch
   User myusername
   GSSAPIAuthentication yes
   GSSAPIDelegateCredentials yes
   Protocol 2
   ForwardX11 no
   IdentityFile ~/.ssh/id_rsa
Host isscvs.cern.ch
   User myusername    
   GSSAPIAuthentication yes
   GSSAPIDelegateCredentials yes
   Protocol 2
   ForwardX11 no
   IdentityFile ~/.ssh/id_rsa

as explained in the SVN How-To.

getpack

After these settings, you are able to get the code (check out) from the repository:

svn co svn+ssh://svn.cern.ch/reps/lhcb/AProject/trunk/MyPackage

svn co svn+ssh://svn.cern.ch/reps/lhcb/AProject/tags/MyPackage/v2r3 MyPackage

on Linux, MacOSX or Windows, however, this is not however the usual way we check out packages in LHCb.

The tool getpack simplifies users' life taking care of details like the presence of several repositories and the project that contains the package. It is also able to do a recursive checkout according to the CMT dependency.

Note that to be able to use getpack you must be able to connect correctly to the Subversion repository without being asked for the password, so be sure that you have the correct configuration explained above.

For backward compatibility with the pre-svn version of getpack, if the environment variable GETPACK_USER is defined, getpack will embed the value of that variable in the URL when checking out the code. Since it is not a good idea, check that you do not set that variable and use the above instructions to use the correct username when accessing the repository.

Guidelines for SVN actions in LHCb

Creating a new package

If you wish to create a new package and add it to Subversion, follow the guidelines in CreateNewPackageSVN.

Guidelines for SVN commit

• Commit often, but commit only code that compiles and runs (i.e. that does not prevent the configuration of the application).
  • Before changing any released package, you should discuss the changes with the package owner (whose name usually appears in the cmt/requirements and doc/release.notes files). It is usually a good idea to inform the appropriate mailing list of the planned changes.
• If this is the first modification to be committed after a release:
  • Increment the version number in the cmt/requirements file
    • Increment the major version (v) if the change is backward incompatible (change in interfaces, major change in behavior)
    • Increment the minor version (r) for a new feature or a bug fix
• Document each commit in the file doc/release.notes
  • Give a brief but meaningful description of the changes ("minor fixes" is not good enough)
    • But no need to document the fix to the fix that you committed a few hours ago
    • Please keep the lines short, max 80 characters per line, carriage return to a new line if necessary.
  • Add your comment above the release separation line (done for you by the <insert> key in the LHCb emacs customisation)

    ! 2009-12-15 - Marco Clemencic - This is the comment for the latest commit, do not touch anything below this line. !================== GaudiConf v10r9 2008-06-04 ========================

• If you are not working with the HEAD revision of a package (i.e. you checked out and modified a tagged version), switch to a directory where you will be able to commit (equivalent of the old cvs update -A preparation before committing):
  • Since TAGGED directories are not supposed to be altered, when committing a change to a package you should, first, switch to the trunk directory. To do so you should
    • Issue the command svn info. That will tell you the actual path of your package:

      svn info
      Path: .
      URL: svn+ssh://svn.cern.ch/reps/lhcb/Rec/tags/Tf/PatAlgorithms/v3r29
      

    • If the URL is of the form above (i.e. it contains the tags subdirectory and a version number), it means you are working against a tagged version. This needs to be altered in order to include trunk in place of tags. You also need to remove the version number at the end of the URL. Do this with the svn switch command:

      svn switch svn+ssh://svn.cern.ch/reps/lhcb/Rec/trunk/Tf/PatAlgorithms
      

• Check what you are about to commit against the repository:

  svn status -u
  

  This will list the files that have changes with respect to the versions in the repository. The type of difference will be marked with one or more letters. The most common are: M for changes in your copy, ? for new files, * for files that have a new version in the repository, C if there are conflicts between your changes and the ones in the repository, ! for files removed. The complete list of letters can be found in the SVN documentation. Once you're sure of your changes, you can call svn update (or svn up) to synchronize the working copy with the changes in repository.
  • Remove any C in front of a file: C in front of a file means that several people are working on the same package so be careful: SVN has tried to merge your modifications with the changes already done and it has NOT succeeded. Check the merged files carefully: SVN marks with >>>>>> and <<<<<< the regions that have incompatible modifications. You must fix these (and remove the <<<<<< and >>>>>> before committing). Merged files only appear when you do svn update, but you then need to be more careful of what you do.
    
  • Remove any ? in source directories: These are files that you added and that are not in the repository. If you wish to save them in SVN, you have to

    svn add TheFileToAdd
    

    do not add any file in the cmt/ directory, or any copy of a file that should not be kept (e.g. files whose name ends with "~")
  • Tell RVN to remove from the repository the files you have deleted if any: files that were marked as lost are files that you removed. If you wish to remove them from CVS, issue the comand

    svn rm TheFileToRemove
    

  • Check again the package:

    svn update
    

    • Any C should have been replaced by M
    • Any ? should have been replaced by A
    • Any ! should have been replaced by R
  • Finally, commit the package

    svn commit
    

    Always add a meaningful comment using the editor that is automatically started.
    You can by-pass the editor with the -m switch:

    svn commit -m "some brief and meaningful comment"
    

• If the package is part of a production project, like Rec, Phys, Analysis etc. then you should add it to the LHCbTagCollector so that the release manager will know what to do with your changes for the release. Use as "tag" the revision number that was printed by svn after the commit. You do not need to provide a "user tag" as was done for CVS. If you do want to provide your own tag, please follow the guidelines for tags
• The next day, check that your commit has not broken other packages
  • Check the nightly builds the day after your commit
  • Fix any packages that were broken by your commit (or ask the package owner to do it)

Guidelines for tags

Users do not need to make tags in SVN, since the revision number provides the same information, but you can, if necessary, or if you are the project manager for example.

Tagging in Subversion, if you are used to another version control system, may appear strange. The concept of tag and branch are not available in Subversion, but they can be easily emulated using a convention, thanks to the ability of Subversion of remembering the history of the whole repository (including directories).

The standard convention used in almost all the projects hosted on Subversion (a notable exception being CMT) is based on the presence of three directories called trunk, tags and branches. The trunk contains the main line of development of your code. tags and branches contains copies of the trunk with symbolic names (like v1r2 or 2.3.4). There is no technical difference between tags and branches, but the convention is that you are not supposed to commit changes to a file in the tags directory.

The convention used for the repository layout in LHCb is described in detail in GaudiSVNRepository.

The guidelines for tagging in LHCb were presented on 18th June 2008 (33rd LHCb software week)

• It is recommended to use a tag of the form <username>_<yyyymmdd> where <yyyymmdd> is the date of the commit. If you have to create more than one user tag during the day, you can append a lowercase letter (starting from a) to distinguish them. To tag the repository as of a certain date (e.g. now) do (note: you can use svn info | awk '/^URL/{print $2}' to get the URL of the package you are using)

  svn cp svn+ssh://svn.cern.ch/reps/lhcb/Brunel/trunk/Rec/Brunel svn+ssh://svn.cern.ch/reps/lhcb/Brunel/tags/Rec/Brunel/marcocle_20091215
  

  or to tag according to what's presently in your directory, do in your directory (e.g. Rec/Brunel):

  svn cp . svn+ssh://svn.cern.ch/reps/lhcb/Brunel/tags/Rec/Brunel/marcocle_20091215
  

  See GaudiSVNRepository for a description of the tags conventions in the LHCb Subversion repository.
• Do not use an official tag of the form vxry

How to alter a tag

Modifying the content of a tag is always a bad policy.

If you really have to do it because after a tag you realized you forgot to do a commit and it is a "user tag" then it is better to create a new tag.

To retag you have to delete the tag you created with something like:

svn rm svn+ssh://svn.cern.ch/reps/lhcb/Brunel/tags/Rec/Brunel/marcocle_20091215

then re-create the tag with the instructions in the previous section.

Subversion documentation

Basic SVN commands

See SVN quick reference card and SVN cheat sheet

A summary of SVN commands for CVS users can be found in the Gaudi twiki.

Some simple instructions to migrate an existing CVS checkout to SVN are available at HowToMigrateACvsCheckoutToSvn.

FAQ

When I try to commit, I get the message "Cannot modify files in tag directories"

I know that I should have the rights to commit, but when I issue the command svn commit src/MuEffMonitor.cpp -m "bla bla" I get the message:

==>>
Sending        src/MuEffMonitor.cpp
Transmitting file data ....svn: Commit faild (details follow):
svn: Commit blocked by pre-commit hook (exit code 1) with output:
Cannot modify files in tag directories

This is because you are not working with the HEAD of the package. You cannot commit to a tag branch, you must commit to the trunk. See the Guidelines for SVN commit

Subversion manual

The complete manual is available from http://svnbook.red-bean.com

Instructions for LHCb librarians

Instructions for LHCb Subversion librarians are available at SubversionSupport.

-- MarcoClemencic - 15-Dec-2009