Difference: DIRAC3SVN (1 vs. 2)

Revision 22009-10-06 - JoelClosier

Line: 1 to 1
 
META TOPICPARENT name="DiracProject"

Dirac SVN Repository

Line: 14 to 14
  Out development model requires the necessity to have package level tags and branches. At the same time, we need something that can be used as a tag on the whole project. Moreover, LHCb unique CVS repository is hosting many CMT projects and a global tag is pointless, but we can profit from the possibility of check out a complete tagged project in one command.
Changed:
<
<

Proposal

The repository contains the 3 main directories packages, projects and distribution.
>
>

Version 2.0

 
Changed:
<
<
packages will contain one directory per package, each of them with the 3 standard directories trunk, tags and branches. In case of CMT "hats", we can have packages/Hat/Package/trunk etc. The list of known packages can be obtained scanning the content of the directory packages, going down one level to take into account the possibility of hats. The scanning is very inefficient, so we can add a property called modules to the packages directory containing a newline-separated list of the known modules. This property can be updated automatically with a cron job or on demand.
>
>

Motivation

Even though the previous structure is valid, it makes working with whole projects more problematic that what it could be. Mainly because of the usage of svn:externals, which is more powerful than the CVS aliases, but it is not flexible enough for our use case (URL pointing to the repository may change).
 
Changed:
<
<
schermata_8.png
>
>
For Gaudi the package tags and the project tags have more or less the same importance (LHCb uses local tags while Atlas and other experiments use only the project tags), and for Dirac (which is being migrated to SVN too) the local tags were never used. This, essentially, means that the projects tags are more important than what was thought when devising the first structure of the repository.
 
Changed:
<
<
The projects directory will contain one directory per hosted project (as container of packages), with the usual 3 standard directories. The content of the trunk for one this projects is only the content of the cmt directory that lives at the project level:
  • projects/DIRAC/trunk/project.cmt
>
>
The current structure makes also problematic to work with RADs like eclipse.

Structure

A structure that allows good integration between project-level and package-level tags can be achieved with something like:

One directory per project (all capital letters, by convention) is visible at the top level of the repository. Inside that directory the three standard directories: trunk, tags and branches. The trunk directory contains the structure of the project as it should be checked out: a cmt directory and one directory per package (with or without a "hat" level).

The tags and branches directories sport the same structure. A directory with the same name of the project will contain the project-level tags/branches, while a directory per package will contain the package-level tags/branches. User tags (or branches) could be stored in further subdirectories in the normal package tags (or branches) directory, for example PROJECT/tags/Package/username/tag.

Not all the foreseen directories are needed for a working repository. A repository can be fully functional without project-level tags or without package-level tags.

In case the grouping of the packages on a per project basis (like in the LHCb repository) is not wanted, a fake project can be used to host all (or part) of the packages (for example called packages). Of course, in this case it is possible to check out a project only using some dedicated tool, like the "getpack" utility used in LHCb.

The special project packages can also be used to host the packages the do not belong (yet/anymore) to a project.

Note that to simplify the maintenance of the repository, when a package needs to be moved from a project to another (or to the special project packages), also the tags and branches directories must be moved.

Meta-data (and getpack)

To make it possible for tools to work with both version 1.0 or version 2.0 of the repository and to take into account the fact that the packages are distributed between various projects (and not grouped in a single directory) some properties of the top-level directory have to be set:
version
must be set to "2.0" to declare the version of the structure of the repository
projects
newline-separated list of projects in the repository, the correct case can should be used
packages
newline-separated list of known packages in the repository
Empty lines or lines starting with # should be ignored.

A line of the packages property must be a space-separated list containing the name of the package (with hat), the name of the project owning it. Optionally it can contain tag specifications (a string representing a path in the repository where %v stands for the version id), to be used to locate special tags (useful if the package has been moved from one project to another). For example:

Hat/PackageA Project   OLDPROJECT/tags/OLDPROJECT/%v/Hat/PackageA
in this case, the tags for Hat/PackageA should be found with the patterns
  • PROJECT/tags/PROJECT/%v/Hat/PackageA
  • PROJECT/tags/Hat/PackageA/%v
  • PROJECT/branches/PROJECT/%v/Hat/PackageA
  • PROJECT/branches/Hat/PackageA/%v
  • OLDPROJECT/tags/OLDPROJECT/%v/Hat/PackageA
 
Deleted:
<
<
The distribution directory contains the 3 standard directories. The tags and branches directory will have one subdirectory per hosted project, each of them containing one directory per "global tag". Those global tag directories will be empty and the internal structure will be described with svn externals definitions. External definitions are metadata of a directory that instruct svn to populate the directory checking out the subdirectories from other svn locations (more or less as cvs module aliases). The trunk directory will contain one empty directory per project, that, with the externals definitions, will allow to check out the trunks of all the related packages in one go.
Note: due to the non-locality of the external definitions (svn allows to check out from different repositories), we cannot use the same distribution sub directory for the public anonymous access and the read/write access, so distribution/trunk/DIRAC should be used by developers, while anonymous users will have to use distribution/tags/DIRAC/DIRAC_HEAD which will checkout the same files as trunk, but using the read-only http protocol.
  schermata_9.png
Line: 33 to 113
 Environment variables for the repositories:
  • DIRACCVS=:kserver:isscvs.cern.ch/local/reps/Dirac
  • DIRACSVN
Changed:
<
<
anonymous read-only access
DIRACSVN=https://svnweb.cern.ch/guest/gaudi
read-write access
DIRACSVN=svn+ssh://svn.cern.ch/reps/gaudi
>
>
anonymous read-only access
DIRACSVN=https://svnweb.cern.ch/guest/dirac
read-write access
DIRACSVN=svn+ssh://svn.cern.ch/reps/dirac
 

Checkout

  • Checkout head version of a package
Line: 44 to 124
 
    • svn

Changed:
<
<
svn checkout $DIRACSVN/packages/DiracConfig/trunk DiracConfig
>
>
svn checkout $DIRACSVN/DIRAC/DiracConfig/trunk/DiracConfig
 
  • Checkout tagged version of a package
    • cvs
Line: 53 to 133
 
    • svn

Changed:
<
<
svn checkout $DIRACSVN/packages/DiracConfig/tags/v20r0 DiracConfig
>
>
svn checkout $DIRACSVN/DIRAC/tags/DiracConfig/v20r0 DiracConfig
 
  • Checkout a complete project head version
    • cvs (Dirac only)
Line: 64 to 144
 
    • svn

Changed:
<
<
svn checkout $DIRACSVN/distribution/trunk/DIRAC
>
>
svn checkout $DIRACSVN/DIRAC/trunk DIRAC
 
    • svn read-only access

Line: 81 to 161
 
mkdir DIRAC
cd DIRAC
Changed:
<
<
svn checkout $DIRACSVN/distribution/tags/DIRAC/DIRAC_v20r0
>
>
svn checkout $DIRACSVN/DIRAC/tags/DIRAC/DIRAC_v20r0
 

Commit

Line: 129 to 209
 
    • svn

Changed:
<
<
svn switch $DIRACSVN/packages/MyPackage/trunk
>
>
svn switch $DIRACSVN/PROJECT/trunk/MyPackage
 
  • Move the working copy from the current tag to the another tag (assuming you are in the top directory of the package)
    • cvs
Line: 138 to 218
 
    • svn

Changed:
<
<
svn switch $DIRACSVN/packages/MyPackage/tags/other_tag
>
>
svn switch $DIRACSVN/PROJECT/tags/MyPackage/other_tag
 

Check for updates in the repository

Line: 172 to 252
 
    • svn

Changed:
<
<
svn diff $DIRACSVN/packages/MyPackage/tags/tag1 $DIRACSVN/packages/MyPackage/tags/tag2
>
>
svn diff $DIRACSVN/PROJECT/tags/MyPackage/tag1 $DIRACSVN/POJECT/tags/MyPackage/tag2
 

Tagging and Branching

Line: 183 to 263
 
    • svn

Changed:
<
<
svn cp . $DIRACSVN/packages/MyPackage/tags/the_tag
>
>
svn cp . $DIRACSVN/PROJECT/tags/MyPackage/the_tag
 
  • Tag the head version in the repository
    • cvs
Line: 192 to 272
 
    • svn

Changed:
<
<
svn cp $DIRACSVN/packages/MyPackage/trunk $DIRACSVN/packages/MyPackage/tags/the_tag
>
>
svn cp $DIRACSVN/PROJECT/trunk/MyPackage $DIRACSVN/PROJECT/tags/MyPackage/the_tag
  Branching is done in CVS addind the option '-b' to the command 'tag', while in Subversion it is enough to replace tags with branches.

Revision 12009-06-08 - JoelClosier

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="DiracProject"

Dirac SVN Repository

Introduction

SVN Repository Organization

Rationale

In a subversion repository, the organization of the paths is of fundamental importance for the everyday work and usability. We want to follow the standard (but not enforced) basic hierarchy of a project hosted on SVN, which consists of the 3 directories:
  • trunk: the main trunk of development
  • tags: fixed versions of the project identified by symbolic names
  • branches: branches of evolution of the project parallel to the trunk

Out development model requires the necessity to have package level tags and branches. At the same time, we need something that can be used as a tag on the whole project. Moreover, LHCb unique CVS repository is hosting many CMT projects and a global tag is pointless, but we can profit from the possibility of check out a complete tagged project in one command.

Proposal

The repository contains the 3 main directories packages, projects and distribution.

packages will contain one directory per package, each of them with the 3 standard directories trunk, tags and branches. In case of CMT "hats", we can have packages/Hat/Package/trunk etc. The list of known packages can be obtained scanning the content of the directory packages, going down one level to take into account the possibility of hats. The scanning is very inefficient, so we can add a property called modules to the packages directory containing a newline-separated list of the known modules. This property can be updated automatically with a cron job or on demand.

schermata_8.png

The projects directory will contain one directory per hosted project (as container of packages), with the usual 3 standard directories. The content of the trunk for one this projects is only the content of the cmt directory that lives at the project level:

  • projects/DIRAC/trunk/project.cmt

The distribution directory contains the 3 standard directories. The tags and branches directory will have one subdirectory per hosted project, each of them containing one directory per "global tag". Those global tag directories will be empty and the internal structure will be described with svn externals definitions. External definitions are metadata of a directory that instruct svn to populate the directory checking out the subdirectories from other svn locations (more or less as cvs module aliases). The trunk directory will contain one empty directory per project, that, with the externals definitions, will allow to check out the trunks of all the related packages in one go.
Note: due to the non-locality of the external definitions (svn allows to check out from different repositories), we cannot use the same distribution sub directory for the public anonymous access and the read/write access, so distribution/trunk/DIRAC should be used by developers, while anonymous users will have to use distribution/tags/DIRAC/DIRAC_HEAD which will checkout the same files as trunk, but using the read-only http protocol.

schermata_9.png

svn Quick Reference for cvs Users

Environment variables for the repositories:
  • DIRACCVS=:kserver:isscvs.cern.ch/local/reps/Dirac
  • DIRACSVN
    anonymous read-only access
    DIRACSVN=https://svnweb.cern.ch/guest/gaudi
    read-write access
    DIRACSVN=svn+ssh://svn.cern.ch/reps/gaudi

Checkout

  • Checkout head version of a package
    • cvs
      cvs -d $DIRACCVS checkout DiracConfig
            
    • svn
      svn checkout $DIRACSVN/packages/DiracConfig/trunk DiracConfig
            
  • Checkout tagged version of a package
    • cvs
      cvs -d $DIRACCVS checkout -r v20r0 DiracConfig
            
    • svn
      svn checkout $DIRACSVN/packages/DiracConfig/tags/v20r0 DiracConfig
            
  • Checkout a complete project head version
    • cvs (Dirac only)
      mkdir DIRAC
      cd DIRAC
      cvs -d $DIRACCVS checkout all
            
    • svn
      svn checkout $DIRACSVN/distribution/trunk/DIRAC
            
    • svn read-only access
      svn checkout $DIRACSVN/distribution/tags/DIRAC/DIRAC_HEAD DIRAC
            
  • Checkout tagged version of a complete project
    • cvs (Dirac only)
      mkdir -p DIRAC/DIRAC_v20r0
      cd DIRAC/DIRAC_v20r0
      cvs -d $DIRACCVS checkout -r DIRAC_v20r0 all
            
    • svn
      mkdir DIRAC
      cd DIRAC
      svn checkout $DIRACSVN/distribution/tags/DIRAC/DIRAC_v20r0
            

Commit

The commit is identical in CVS and Subversion
  • Commit the changes in the current directory (and subdirectories)
    • cvs
      cvs commit 
            
    • svn
      svn commit
            
  • Commit the changes of a single file
    • cvs
      cvs commit "myfile"
            
    • svn
      svn commit "myfile"
            

Update

I want to distinguish between two use cases:
  • simple update: synchronize the local copy with the latest (or a specific) revision of the repository
  • update with tags: move from a tagged local version to the head one or to a branch

Simple Update

Nothing really special:
  • cvs
    cvs update
       
  • svn
    svn update
       
The way to select a revision or a moment in time a slightly different, but easy to sort out using the help.

Update with Tags

This is the complex part, since tags do not exists (as such) in Subversion. With SVN, tags are represented by copies of the trunk directory, so moving the working copy from a tag to another (or trunk or branch) means changing the path of the origin of the working copy.
  • Move the working copy from the current tag to the trunk/HEAD (assuming you are in the top directory of the package)
    • cvs
      cvs update -APd
            
    • svn
      svn switch $DIRACSVN/packages/MyPackage/trunk
            
  • Move the working copy from the current tag to the another tag (assuming you are in the top directory of the package)
    • cvs
      cvs update -Pd -r other_tag
            
    • svn
      svn switch $DIRACSVN/packages/MyPackage/tags/other_tag
            

Check for updates in the repository

To just check if there are changes in the repository that you didn't get yet, you can do a "dry-run" update with cvs. That does not exist in svn, you have to ask the status of the repository.
  • cvs
    cvs update -n
          
  • svn
    svn status -u
          
    or
    svn status --show-updates
          

Comparison (diff)

  • Comparison between the base of the working copy and the working copy
    • cvs
      cvs diff -u
            
    • svn
      svn diff
            
  • Comparison between two tags (or branch, or trunk)
    • cvs
      cvs diff -u -r tag1 -r tag2
            
    • svn
      svn diff $DIRACSVN/packages/MyPackage/tags/tag1 $DIRACSVN/packages/MyPackage/tags/tag2
            

Tagging and Branching

  • Tag the working copy (assuming you are in the top directory of the package)
    • cvs
      cvs tag the_tag
            
    • svn
      svn cp .  $DIRACSVN/packages/MyPackage/tags/the_tag
            
  • Tag the head version in the repository
    • cvs
      cvs rtag the_tag MyPackage
            
    • svn
      svn cp $DIRACSVN/packages/MyPackage/trunk $DIRACSVN/packages/MyPackage/tags/the_tag
            
Branching is done in CVS addind the option '-b' to the command 'tag', while in Subversion it is enough to replace tags with branches.

References

Help desk reported problems

These are just to remember them.

-- Main.JoelClosier - 08 Jun 2009

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback