SVN repository access:

svn co https://svn.cern.ch/reps/dashboard/trunk -- check out current development version

Branches are in https://svn.cern.ch/reps/dashboard/branches , tags are in https://svn.cern.ch/reps/dashboard/tags

URL for anonymous read-only access: http://svnweb.cern.ch/guest/dashboard/

Repository structure

• /trunk
  • arda.dashboard/
  • arda.dashboard.agis/
  • arda.dashboard.apel/
  • ....
  • client-gwt/
  • ....
  • server-proddb/
  • server-storage/

• /branches
  • arda.dashboard/
  • arda.dashboard.agis/
    • dashboard-agis-0-1-0-branch/
      • arda.dashboard.agis/
  • arda.dashboard.apel/
    • dashboard-apel-0-1-0-branch/
      • arda.dashboard.apel/
    • dashboard-apel-0-2-0-branch/
      • arda.dashboard.apel/
  • .....

• /tags
  • arda.dashboard/
    • dashboard-0-0-0_rc1/
      • arda.dashboard/
    • dashboard-0-0-0_rc2/
      • arda.dashboard/
  • arda.dashboard.agis/
    • dashboard-agis-0-1-0/
      • arda.dashboard.agis/
    • dashboard-agis-0-1-0_rc1/
      • arda.dashboard.agis/
    • .....
  • arda.dashboard.apel/
    • dashboard-apel-0-1-0/
      • arda.dashboard.apel
    • dashboard-apel-0-1-0_rc1/
      • arda.dashboard.apel
    • .....
  • .....

Commit rights

Current repository access rights are declared here.

Versioning of repository's administrative files

Configuration files of the project's SVN repository are located in AFS in /afs/cern.ch/project/svn/reps/dashboard/conf , user hooks handlers are in /afs/cern.ch/project/svn/reps/dashboard/usr-hooks . They are accessible on behalf of special librarian user, libdashb.

File /afs/cern.ch/project/svn/reps/dashboard/conf/authz contains authoarization and access rights information, /afs/cern.ch/project/svn/reps/dashboard/conf/mailer.conf is a configuration file for mail notifiactions on actions like commit.

Files in these directories could be changed by librarian, so they also should be versioned as other files of the project.

Proper place to keep versions of the directories is https://svn.cern.ch/reps/admin/dashboard

The directory to version project's settings is accessible with librarian's account ( libdashb):

svn --username libdashb co https://svn.cern.ch/reps/admin/dashboard

It is to be managed as a normal SVN directory.

Typical administrative actions

Create a new module

Go to the directory where you have checked out the whole repository

svn co https://svn.cern.ch/reps/dashboard
cd dashboard

Choose a module name that describes what it does (e.g. if it is a web interface, do something like 'web-...', ) Create a directory for that module under trunk, branches and tags

svn mkdir trunk/arda.dashboard.#modulename#
svn mkdir tags/arda.dashboard.#modulename#
svn mkdir branches/arda.dashboard.#modulename#
svn commit 

Grant access to a user

To grant access to a user:

• Login as libdashb
• Modify the authz
• Put the afs login of the user under the module, in the trunk, branches and tags
• If the module does not exist, create it.

For instance, to give access to bob to module 'arda.dashboard.bobmodule'

ssh libdashb@lxplus
vi $HOME/repo/conf/authz
# Add  the lines  (without the comments)

# [/trunk/arda.dashboard.bobmodule]
# bob = rw
#[/tags/arda.dashboard.bobmodule]
#i bob = rw
#[/branches/arda.dashboard.bobmodule]
# bob= rw

Turn off the pre-commit rules

If we desperately need to by-pass the dashboard rules for SVN commits then you have to:

• Login as libdashb
• Modify the pre-commit rules:

ssh libdashb@lxplus 
vi  /afs/cern.ch/user/l/libdashb/repo/usr-hooks/pre-commit
#Comment out the line  $DASHBOARD "$REPOS" "$TXN" || exit 1

Migration from CVS to SVN

Migrate the repository data

• Connect to the dedicated server lxsvn with the appropriate version of SVN client (1.5, as on centrel SVN server svn.cern.ch)

ssh libdashb@lxsvn

• Make temporary directory to keep repositories dump files and copies

mkdir /tmp/dashboard-migration

cd /tmp/dashboard-migration

• Dump current content of dashboard repository

svnadmin dump /afs/cern.ch/project/svn/reps/dashboard/ > dashb-svn-repo-backup

• Make a local copy of Dashbord CVS repository to the directory dashboard-cvs

cp -a /net/dashboard.cvs.cern.ch/local/projects/dashboard/rep dashboard-cvs

• Dump the content of CVS repository from directory dashboard-cvs, commit by commit, to the format accepted by SVN (file will be named dashboard-cvs-dump). Head branch will be placed to /trunk directory, branches and tags from CVS will appear in SVN's directories /branches and /tags respectively.

/afs/cern.ch/project/svn/cvs2svn/cvs2svn-2.1.1/cvs2svn --use-cvs --fallback-encoding=ascii --dumpfile=dashboard-cvs-dump --trunk=/trunk --branches=/branches --tags=/tags dashboard-cvs

• Directories /trunk, /brances and tags already exist in the repository, so in the dump fille dashboard-cvs-dump creation of such directories should be prevented. For example, one of the easiest ways is to rename them and then delete after loading the dump into th repository.

Node-path: trunk
Node-kind: dir
Node-action: add


Node-path: branches
Node-kind: dir
Node-action: add


Node-path: tags
Node-kind: dir
Node-action: add

Should be changed to

Node-path: trunk-tmp
Node-kind: dir
Node-action: add


Node-path: branches-tmp
Node-kind: dir
Node-action: add


Node-path: tags-tmp
Node-kind: dir
Node-action: add

• Load dumped CVS data into SVN repository. Just applies all dumped commits one by one. All data which was in destination repository is preserved.
  

svnadmin load /afs/cern.ch/project/svn/reps/dashboard < dashboard-cvs-dump

• Remove temporary directories /branches-tmp, /tags-tmp, /trunk-tmp (see above)

svn co --depth=immediates https://svn.cern.ch/reps/dashboard/

cd dashboard

svn rm branches-tmp tags-tmp trunk-tmp

svn ci -m "Auxiliary directories were removed"

Update scripts and configuration files

List of affected files:

• Modified

arda.dashboard/config/build/dashboard-testing.xml
arda.dashboard/config/build/dashboard-unstable.xml
arda.dashboard/config/build/dashboard-nightly.xml
arda.dashboard/config/build/dashboard-nightly.cron
arda.dashboard/config/build/dashboard-cms.xml
arda.dashboard/config/build/dashboard-stable.xml
arda.dashboard/config/pylint/dashb.rc
arda.dashboard/lib/dashboard/distutils/load.py
arda.dashboard/lib/dashboard/distutils/release.py
arda.dashboard/lib/dashboard/distutils/fullbuild.py
arda.dashboard/lib/dashboard/distutils/doc.py

• Added

arda.dashboard/config/build/dashboard-build-0.2.0.xsd

arda.dashboard/bin/svn2cl.sh
arda.dashboard/bin/svn2cl.xsl

svn2cl could be downloaded from its official site: http://arthurdejong.org/svn2cl/downloads.html . Only files svn2cl.sh and svn2cl.xsl are required.

• Removed

arda.dashboard/bin/cvs2cl.pl

Ignoring rules for files and directories

Set ignore rules recursively:

svn propset -R svn:ignore "*.pyc
*.pyo
*.swp" '.'

svn ci -m "Adding common ignore rules recursively"

Rename files .cvsignore to .svnignore, then apply rules from .svnignore to the directories which contain them.

To fild and rename .cvsignore following script was used:

#!/bin/bash

base_dir=`pwd`

for f in `find ${base_dir} -name ".cvsignore"`; do
dir=`dirname $f`
cd $dir
svn mv .cvsignore .svnignore
svn propset svn:ignore -F .svnignore '.'
cd $base_dir
done

Then commit the changes:

svn ci -m ".cvsignore files renamed to .svnignore, then svn:ignore properties applied to corresponding directories"

Access rights settings

THe main place for authorization settings is /afs/cern.ch/project/svn/reps/dashboard/conf/authz . For extended access rights setup pre-commit hook scripts should be used (see below).

TODO

• Access control for tags and branches

svnperms.py or commit-access-control.pl (subversion tools) could be used to preform pre-commit access rights resolution. In the main authorization file, authz, it is not possible to use masks for directories' path, therefore an addidional check is required for branches and tags.

• Mail notifications customization

Extended settings for mailer.py in /afs/cern.ch/project/svn/reps/dashboard/conf/mailer.conf (e.g. per-module notifications)

(detailed description of mailer module setup is here: http://opensource.perlig.de/svnmailer/doc-1.0/)

Links

Many useful tips on managing and tuning SVN repository could be found on CERN SVN Howto page

Nice CVS to SVN notes by Ricardo Rocha

-- SergeyBelov - 14-Apr-2010

• dashb-svn-migration.pdf: Presentation on CVS to SVN migration (from Dashboard group meeting)

SVN repository access:

svn co https://svn.cern.ch/reps/dashboard/trunk -- check out current development version

Branches are in https://svn.cern.ch/reps/dashboard/branches , tags are in https://svn.cern.ch/reps/dashboard/tags

URL for anonymous read-only access: http://svnweb.cern.ch/guest/dashboard/
Repository structure

/branches


ARDA/
arda/
dashboard-agis-0-1-0-branch/
dashboard-apel-0-1-0-branch/
dashboard-apel-0-2-0-branch/
dashboard-api-0-1-0-branch/


....


/tags


Root_dashboard-dao-oracle-job-production-lu-feng/
Root_dashboard-dao-oracle-job-production-only-one-DAO-file-branch/
Root_dashboard-util-plotting-branch-matplotlib/
Root_dashboard-web-job-multiple-vo/


....


/trunk


arda.dashboard/
arda.dashboard.agis/
arda.dashboard.apel/


....

client-gwt/
....
server-proddb/
server-storage/



Versioning of repository's administrative files

Configuration files of the project's SVN repository are located in AFS in /afs/cern.ch/project/svn/reps/dashboard/conf , user hooks handlers are in /afs/cern.ch/project/svn/reps/dashboard/usr-hooks . They are accessible on behalf of special librarian user, libdashb.

File /afs/cern.ch/project/svn/reps/dashboard/conf/authz contains authoarization and access rights information, /afs/cern.ch/project/svn/reps/dashboard/conf/mailer.conf is a configuration file for mail notifiactions on actions like commit.

Files in these directories could be changed by librarian, so they also should be versioned as other files of the project.

Proper place to keep versions of the directories is https://svn.cern.ch/reps/admin/dashboard

The directory to version project's settings is accessible with librarian's account ( libdashb):

svn --username libdashb co https://svn.cern.ch/reps/admin/dashboard


It is to be managed as a normal SVN directory.
Migration from CVS to SVN
Migrate the repository data

* Connect to the dedicated server lxsvn with the appropriate version of SVN client (1.5, as on centrel SVN server svn.cern.ch)

ssh libdashb@lxsvn


* Make temporary directory to keep repositories dump files and copies

mkdir /tmp/dashboard-migration

cd /tmp/dashboard-migration



* Dump current content of dashboard repository


svnadmin dump /afs/cern.ch/project/svn/reps/dashboard/ > dashb-svn-repo-backup



* Make a local copy of Dashbord CVS repository to the directory dashboard-cvs

cp -a /net/dashboard.cvs.cern.ch/local/projects/dashboard/rep dashboard-cvs


* Dump the content of CVS repository from directory dashboard-cvs, commit by commit, to the format accepted by SVN (file will be named dashboard-cvs-dump). Head branch will be placed to /trunk directory, branches and tags from CVS will appear in SVN's directories /branches and /tags respectively.


/afs/cern.ch/project/svn/cvs2svn/cvs2svn-2.1.1/cvs2svn --use-cvs --fallback-encoding=ascii --dumpfile=dashboard-cvs-dump --trunk=/trunk --branches=/branches --tags=/tags dashboard-cvs



* Directories /trunk, /brances and tags already exist in the repository, so in the dump fille dashboard-cvs-dump creation of such directories should be prevented. For example, one of the easiest ways is to rename them and then delete after loading the dump into th repository.


Node-path: trunk
Node-kind: dir
Node-action: add


Node-path: branches
Node-kind: dir
Node-action: add


Node-path: tags
Node-kind: dir
Node-action: add


Should be changed to


Node-path: trunk-tmp
Node-kind: dir
Node-action: add


Node-path: branches-tmp
Node-kind: dir
Node-action: add


Node-path: tags-tmp
Node-kind: dir
Node-action: add



* Load dumped CVS data into SVN repository. Just applies all dumped commits one by one. All data which was in destination repository is preserved.


svnadmin load /afs/cern.ch/project/svn/reps/dashboard < dashboard-cvs-dump



* Remove temporary directories /branches-tmp, /tags-tmp, /trunk-tmp (see above)

svn co --depth=immediates https://svn.cern.ch/reps/dashboard/

cd dashboard

svn rm branches-tmp tags-tmp trunk-tmp

svn ci -m "Auxiliary directories were removed"



Update scripts and configuration files

List of affected files:

* Modified

arda.dashboard/config/build/dashboard-testing.xml
arda.dashboard/config/build/dashboard-unstable.xml
arda.dashboard/config/build/dashboard-nightly.xml
arda.dashboard/config/build/dashboard-nightly.cron
arda.dashboard/config/build/dashboard-cms.xml
arda.dashboard/config/build/dashboard-stable.xml
arda.dashboard/config/pylint/dashb.rc
arda.dashboard/lib/dashboard/distutils/load.py
arda.dashboard/lib/dashboard/distutils/release.py
arda.dashboard/lib/dashboard/distutils/fullbuild.py
arda.dashboard/lib/dashboard/distutils/doc.py

* Added

arda.dashboard/config/build/dashboard-build-0.2.0.xsd

arda.dashboard/bin/svn2cl.sh
arda.dashboard/bin/svn2cl.xsl


svn2cl could be downloaded from its official site: http://arthurdejong.org/svn2cl/downloads.html . Only files svn2cl.sh and svn2cl.xsl are required.


* Removed

arda.dashboard/bin/cvs2cl.pl

Ignoring rules for files and directories

Set ignore rules recursively:


svn propset -R svn:ignore "*.pyc
*.pyo
*.swp" '.'


svn ci -m "Adding common ignore rules recursively"


Rename files .cvsignore to .svnignore, then apply rules from .svnignore to the directories which contain them.

To fild and rename .cvsignore following script was used:


#!/bin/bash

base_dir=`pwd`

for f in `find ${base_dir} -name ".cvsignore"`; do
dir=`dirname $f`
cd $dir
svn mv .cvsignore .svnignore
svn propset svn:ignore -F .svnignore '.'
cd $base_dir
done


Then commit the changes:


svn ci -m ".cvsignore files renamed to .svnignore, then svn:ignore properties applied to corresponding directories"



Access rights settings

THe main place for authorization settings is /afs/cern.ch/project/svn/reps/dashboard/conf/authz . For extended access rights setup pre-commit hook scripts should be used (see below).
TODO

* Access control for tags and branches

svnperms.py or commit-access-control.pl (subversion tools) could be used to preform pre-commit access rights resolution. In the main authorization file, authz, it is not possible to use masks for directories' path, therefore an addidional check is required for branches and tags.

* Mail notifications customization

Extended settings for mailer.py in /afs/cern.ch/project/svn/reps/dashboard/conf/mailer.conf (e.g. per-module notifications)
Links

Many useful tips on managing and tuning SVN repository could be found on CERN SVN Howto page

Nice CVS to SVN notes by Ricardo Rocha

-- SergeyBelov - 14-Apr-2010


* dashb-svn-migration.pdf: Presentation on CVS to SVN migration (from Dashboard group meeting)