Difference: LHCbRichMirrorAlignGitInfo (1 vs. 35)

Revision 352018-07-03 - AnatolySolomin

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 110 to 110
 Only once and forever do these three things:
  • Follow Prerequisites
  • git config --global user.name "Anatoly Solomin" (where Anatoly Solomin = your name)
Changed:
<
<
>
>
  These should be done every time you want to start coding (NOTE the choice of User_release_area is completely up to you, actually, but should be in your AFS area):
Changed:
<
<
  • mkdir /afs/cern.ch/user/a/asolomin/gituser (where asolomin = your username)
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser (where asolomin = your username)
>
>
  • mkdir -p /afs/cern.ch/user/a/asolomin/public/rich_align (where asolomin = your username)
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/public/rich_align (where asolomin = your username)
 
  • LbLogin (for some reason you have to run this after setting the user release area)
  • cd $User_release_area
Changed:
<
<
Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here we assume the newest is v8r1, you should check in gitlab):
  • lb-dev --nightly lhcb-head Panoptes/v8r1
    • This creates a local git repository in PanoptesDev_v8r1. This is called a "satellite project".
    • (NOTE we do need to use the nightlies, unless we know that other people are working on code that may affect us... simply use lb-dev Panoptes/v8r1 instead if this is not the immediate case)
>
>
Then this you do only once, when you start from scratch (though perform it separately when starting from each new Panoptes release, here we assume we are working with 2018-patches):
  • lb-dev --nightly-cvmfs --nightly lhcb-2018-patches Panoptes/2018-patches
    • This creates a local git repository in PanoptesDev_2018-patches. This is called a "satellite project".
    • (NOTE we do need to use the nightlies, unless we know that other people are working on code that may affect us... simply use lb-dev Panoptes/2018-patches instead if this is not the immediate case)
  These you do as long as you are working with the project and version in the previous line (NOTE the "Dev"):
Changed:
<
<
  • cd $User_release_area/PanoptesDev_v8r1
>
>
  • cd $User_release_area/PanoptesDev_2018-patches
 
  • git lb-use Panoptes
    • This adds the gitlab remote to the local repository of the satellite project, and fetches that remote (makes a local copy of the remote repository, although you still don't see it in your working tree).
Line: 193 to 193
 You should try to make a new commit every time you've made modifications that can be considered a single unit of changes.

You will now give your branch a name. You could choose anything (but pick something unique!). However let's try to stick to the following prescription:

Changed:
<
<
username-title-YYYYMMDD (e.g. asolomin-MirrAlign-20170123).
So for example, to push your changes to a new branch called asolomin-MirrAlign-20170123:
  • git lb-push Panoptes asolomin-MirrAlign-20170123
>
>
username-title-YYYYMMDD (e.g. asolomin-MirrAlign-20180703).
So for example, to push your changes to a new branch called asolomin-MirrAlign-20180703:
  • git lb-push Panoptes asolomin-MirrAlign-20180703
  Here, your equivalent of username-title-YYYYMMDD is the name of your development branch, and you will be able to see it in gitlab.

If you then would like the commits in username-title-YYYYMMDD to be merged into 2018-patches, you must perform a Gitlab merge request.

Changed:
<
<
You may submit a merge request for asolomin-MirrAlign-20170123 from the Panoptes gitlab page
>
>
You may submit a merge request for asolomin-MirrAlign-20180703 from the Panoptes gitlab page
 You can also use the hyperlink provided when you commit, though if it's a sensitive change you may want to look at your commit in gitlab first. NOTE: Your pushed branch (that contains your commit) MUST COMPILE locally to get accepted (though it may not compile against the nightlies, consult Chris and Jordi if you find this problem).

Important: if you noticed a mistake, fix it, and immediately want to commit a corrected version to a new branch, you should

Changed:
<
<
  • git lb-push Panoptes asolomin-MirrAlign-20170123-2 (i.e. any different name)
Then, submit a merge request for asolomin-MirrAlign-20170123-2, instead of asolomin-MirrAlign-20170123
>
>
  • git lb-push Panoptes asolomin-MirrAlign-20180703-2 (i.e. any different name)
Then, submit a merge request for asolomin-MirrAlign-20180703-2, instead of asolomin-MirrAlign-20180703
 Go ahead and delete the branch.

Now say you already made a merge request, then you could commit a fix and push it to the same branch for which you made the merge request. Before you start work on the fix however, make sure to change the first word of the title of your merge request to "WIP: " (work-in-progress). This instructs the Project manager to wait until you remove "WIP: " to merge the branch.

Line: 218 to 218
 This shouldn't happen in our workflow, but just so you know about it... If changes were made to your branch [e.g. in GitLab] by someone else, and then you want to keep working on the branch:
  • git fetch --all # (you must do this in order to get access to remote changes in Panoptes/2018-patches or any other branch)
Changed:
<
<
  • git lb-checkout Panoptes/asolomin-MirrAlign-20170123 Rich/RichMirrorAlignmentOnline
>
>
  • git lb-checkout Panoptes/asolomin-MirrAlign-20180703 Rich/RichMirrorAlignmentOnline
  After the merge requests are applied and only after, when you are ready to start work again you will want to update your local packages from the 2018-patches, in case there were any changes in the 2018-patches in the meantime:
Changed:
<
<
  • cd $User_release_area/PanoptesDev_v8r1
>
>
  • cd $User_release_area/PanoptesDev/2018-patches
 
  • git fetch --all # (Remember, you must do this in order to get access to remote changes in Panoptes/2018-patches or any other branch)
  • git lb-checkout Panoptes/2018-patches Rich/RichMirrorAlignmentOnline (and the same for ALL other packages you checked out from 2018-patches, unless you are still working on a branch of a different package)
Line: 247 to 247
 Say you wanted to run a test of your code. Right now we can only test Rich/RichMirrorAlignmentOnline on the farm. But if you edited Rich/RichMirrAlign you could test it in the following manner:

new shell ( e.g. ssh -Y asolomin@lxplusNOSPAMPLEASE.cern.ch )

Changed:
<
<
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser
>
>
  • ssh lbgw -Y
  • cd /group/online/AligWork/MirrorAlignments
  • scp -pr Rich1/20180628_172734 asolomin@lxplusNOSPAMPLEASE.cern.ch:/afs/cern.ch/user/a/asolomin/public/rich_align_test/Rich1/
  • exit
  • cd /afs/cern.ch/user/a/asolomin/public/rich_align_test/Rich1/
  • cp -pr 20180628_172734 20180628_172734_test
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/public/rich_align
 
  • LbLogin
Changed:
<
<
  • cd $User_release_area/PanoptesDev_v8r1
>
>
  • cd $User_release_area/PanoptesDev/2018-patches
 
  • make install
Changed:
<
<
>
>
 

Slight differences to the above for the Hlt project

Revision 342018-05-23 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 118 to 118
 
  • LbLogin (for some reason you have to run this after setting the user release area)
  • cd $User_release_area
Changed:
<
<
Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here we assume the newest is v7r3, you should check in gitlab):
  • lb-dev --nightly lhcb-head Panoptes/v7r3
    • This creates a local git repository in PanoptesDev_v7r3. This is called a "satellite project".
    • (NOTE we do need to use the nightlies, unless we know that other people are working on code that may affect us... simply use lb-dev Panoptes/v7r3 instead if this is not the immediate case)
>
>
Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here we assume the newest is v8r1, you should check in gitlab):
  • lb-dev --nightly lhcb-head Panoptes/v8r1
    • This creates a local git repository in PanoptesDev_v8r1. This is called a "satellite project".
    • (NOTE we do need to use the nightlies, unless we know that other people are working on code that may affect us... simply use lb-dev Panoptes/v8r1 instead if this is not the immediate case)
  These you do as long as you are working with the project and version in the previous line (NOTE the "Dev"):
Changed:
<
<
  • cd $User_release_area/PanoptesDev_v7r3
>
>
  • cd $User_release_area/PanoptesDev_v8r1
 
  • git lb-use Panoptes
    • This adds the gitlab remote to the local repository of the satellite project, and fetches that remote (makes a local copy of the remote repository, although you still don't see it in your working tree).
Line: 134 to 134
  * [new branch] 2018-patches -> Panoptes/2018-patches
* [new tag] v1r0 -> Panoptes/v1r0
[...]
Changed:
<
<
* [new tag] v7r1 -> Panoptes/v7r1
* [new tag] v7r3 -> Panoptes/v7r3
>
>
* [new tag] v8r1 -> Panoptes/v8r1
 

You will see that there are several references that you can use.

Line: 222 to 221
 
  • git lb-checkout Panoptes/asolomin-MirrAlign-20170123 Rich/RichMirrorAlignmentOnline

After the merge requests are applied and only after, when you are ready to start work again you will want to update your local packages from the 2018-patches, in case there were any changes in the 2018-patches in the meantime:

Changed:
<
<
  • cd $User_release_area/PanoptesDev_v7r3
>
>
  • cd $User_release_area/PanoptesDev_v8r1
 
  • git fetch --all # (Remember, you must do this in order to get access to remote changes in Panoptes/2018-patches or any other branch)
  • git lb-checkout Panoptes/2018-patches Rich/RichMirrorAlignmentOnline (and the same for ALL other packages you checked out from 2018-patches, unless you are still working on a branch of a different package)
Line: 250 to 249
 new shell ( e.g. ssh -Y asolomin@lxplusNOSPAMPLEASE.cern.ch )
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser
  • LbLogin
Changed:
<
<
  • cd $User_release_area/PanoptesDev_v7r3
>
>
  • cd $User_release_area/PanoptesDev_v8r1
 
  • make install
  • cd /afs/cern.ch/user/a/asolomin/public/rich_align_test (or wherever your test is)
Changed:
<
<
>
>
 

Slight differences to the above for the Hlt project

Revision 332018-04-26 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 27 to 27
 The basic idea behind git is that "branches" are used to develop new features of the code, typically each branch forks from the main master branch (the "default" branch) of a git repository, and each new branch is isolated from the other branches, until one chooses to merge them. Typically we use these other branches for development, and then merge them back to the master branch when the new feature is tested and ready. At LHCb we have to ask for a merge request in CERN's GitLab to merge one of our branches into (typically) the master branch of a Project. Then the Project manager can decide whether to allow the merge to occur.

NOTA BENE: However, Panoptes is currently using two main branches. One is called 2018-patches and applies only to 2018 operations, the other is called master and will be used in the upgrade. This significantly changes our workflow:

Changed:
<
<
  • Always lb-checkout out code from 2018-patches and commit any changes to a new branch
  • As usual, set up a merge request when ready to merge the new branch into the 2018-patches branch.
  • NEW Cherry-pick commits from 2018-patches into a second new branch.
  • Set up a merge request when ready to merge the second new branch into the master branch.
>
>
  • ALWAYS lb-checkout out code from 2018-patches and commit any changes to a new branch
  • As usual, set up a merge request when ready to merge the new branch and make CERTAIN that you are merging into the 2018-patches branch.
  • Within the discussion section of the merge request, kindly ask Jordi to rebase these commits into the master branch once the merge request is approved.
    • Jordi will happily do this so long as there are no conflicts. If there is a conflict then you will have to resolve it because "the author knows best".
      • To solve it, you may have to cherry-pick commits from 2018-patches into a second new branch (see below), and then set up a merge request when ready to merge the second new branch into the master branch.
 This seems complicated but it's the only way we can ensure that our changes go into both branches, as the master branch will soon have much code stripped away and changed in favor of the upgrade era code.

Standard documentation and other git stuff

Line: 283 to 285
 

Cherry-picking commits from 2018-patches into master

Changed:
<
<
In the new workflow, we always work with 2018-patches as our starting branch, and we will use 2018-patches as the default branch at the pit as well. However, since for the time being we also want the upgrade code to have all of the changes we made, we are required to manually cherry-pick commits we make to 2018-patches to a new branch, and then merge that new branch into master.
>
>
In the new workflow, we always work with 2018-patches as our starting branch, and we will use 2018-patches as the default branch at the pit as well. Thus make sure that your merge requests are to merge your branch into 2018-patches. However, since for the time being we also want the upgrade code to have all of the changes we made, we are required to either:

1. Rebase the commits that are in your merge request into master

or

2. Manually cherry-pick commits we make to 2018-patches to a new branch, and then merge that new branch into master.

 
Changed:
<
<
There are several ways to do this, which will not be discussed here. However this is something that you have to do, so if you can't figure it out please contact all the mirror alignment experts for advice.
>
>
The former can be done by Jordi as long as there are no merge conflicts (see above on how to request that Jordi do this for you). There are several ways to do the latter, which will not be discussed here. However may be something that you have to do, so if you can't figure it out please contact all the mirror alignment experts for advice.
 

Revision 322018-02-12 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 24 to 24
  It is time to git! Please read everything and please follow the workflow Anatoly describes. If you have any questions email the alignment-development mailing list. If we cannot answer your question, email Jordi, and if all else fails then Marco Clemencic / lhcb-core-soft@cernNOSPAMPLEASE.ch
Changed:
<
<
The basic idea behind git is that "branches" are used to develop new features of the code, each branch forks from the master branch (the "default" branch) of a git repository, and each new branch is isolated from the other branches, until one chooses to merge them. Typically we use these other branches for development, and then merge them back to the master branch when the new feature is tested and ready. At LHCb we have to ask for a merge request in CERN's GitLab to merge one of our branches into the master branch of a Project. Then the Project manager can decide whether to allow the merge to occur.
>
>
The basic idea behind git is that "branches" are used to develop new features of the code, typically each branch forks from the main master branch (the "default" branch) of a git repository, and each new branch is isolated from the other branches, until one chooses to merge them. Typically we use these other branches for development, and then merge them back to the master branch when the new feature is tested and ready. At LHCb we have to ask for a merge request in CERN's GitLab to merge one of our branches into (typically) the master branch of a Project. Then the Project manager can decide whether to allow the merge to occur.

NOTA BENE: However, Panoptes is currently using two main branches. One is called 2018-patches and applies only to 2018 operations, the other is called master and will be used in the upgrade. This significantly changes our workflow:

  • Always lb-checkout out code from 2018-patches and commit any changes to a new branch
  • As usual, set up a merge request when ready to merge the new branch into the 2018-patches branch.
  • NEW Cherry-pick commits from 2018-patches into a second new branch.
  • Set up a merge request when ready to merge the second new branch into the master branch.
This seems complicated but it's the only way we can ensure that our changes go into both branches, as the master branch will soon have much code stripped away and changed in favor of the upgrade era code.
 

Standard documentation and other git stuff

Line: 51 to 58
 
  • However, packages don't have version tags anymore. From now on, a Panoptes project tag affects all its packages. What used to be "version v2r26 of Rich/RichOnlineMonitors" is now the "Rich/RichOnlineMonitors that comes with Panoptes v5r6."
  • The tag collector becomes obsolete from the point of view of Panoptes.
Changed:
<
<
  • Since we still need to release Panoptes versions, all packages should at least compile when they are added to the master branch of the git repository.
>
>
  • Since we still need to release Panoptes versions, all packages should at least compile when they are added to the 2018-patches / master branch of the git repository.
  For this reason, it is important to have meaningful development branches for stuff that is not yet stable, which will be merged once the new features are fully implemented and tested.
Line: 60 to 67
 
  • Implement the features in a branch.
  • Test them.
  • Push the branch to an upstream repository (e.g. a fork of the project in gitlab).
Changed:
<
<
  • Submit a merge request, so the new features are included in the master branch.
>
>
  • Submit a merge request, so the new features are included in the 2018-patches / master branch.
  Please get used to the git workflow ASAP.
Line: 122 to 129
 The output of this last command will give you information like this:
 * [new branch]      master     -> Panoptes/master <br>
Added:
>
>
* [new branch] 2018-patches -> Panoptes/2018-patches
  * [new tag] v1r0 -> Panoptes/v1r0
[...]
* [new tag] v7r1 -> Panoptes/v7r1
Line: 129 to 137
 

You will see that there are several references that you can use.

Changed:
<
<
Some of them are branches (in this example, only master) and some of them are tags.
>
>
Some of them are branches (in this example, 2018-patches and master) and some of them are tags.
 You will refer to them as Panoptes/[reference], as in the last column of the previous output.
Changed:
<
<
Let's say you want to develop on top of the Rich/RichMirrorAlignmentOnline package that you can find on the master branch.
>
>
Let's say you want to develop on top of the Rich/RichMirrorAlignmentOnline package that you can find on the 2018-patches branch.
 To checkout a package (e.g before you start work on a new package for the first time):
  • git fetch --all (ALWAYS do this before any lb-checkout, it updates the local information about branches with the current information from the git repository.)
Changed:
<
<
  • git lb-checkout Panoptes/master Rich/RichMirrorAlignmentOnline (or perhaps a different package that you might be working on (e.g. Rich/RichMirrAlign))
>
>
  • git lb-checkout Panoptes/2018-patches Rich/RichMirrorAlignmentOnline (or perhaps a different package that you might be working on (e.g. Rich/RichMirrAlign))
 
    • This is the git equivalent of checking out the package from the SVN head.

Now you may edit the code and add new features. You would obviously not necessarily edit the same files as shown in this example, but whatever pieces of code you actually wanted to edit.

Line: 190 to 198
  Here, your equivalent of username-title-YYYYMMDD is the name of your development branch, and you will be able to see it in gitlab.
Changed:
<
<
If you then would like the commits in username-title-YYYYMMDD to be merged into master, you must perform a Gitlab merge request.
>
>
If you then would like the commits in username-title-YYYYMMDD to be merged into 2018-patches, you must perform a Gitlab merge request.
 You may submit a merge request for asolomin-MirrAlign-20170123 from the Panoptes gitlab page You can also use the hyperlink provided when you commit, though if it's a sensitive change you may want to look at your commit in gitlab first. NOTE: Your pushed branch (that contains your commit) MUST COMPILE locally to get accepted (though it may not compile against the nightlies, consult Chris and Jordi if you find this problem).
Line: 208 to 216
  This shouldn't happen in our workflow, but just so you know about it... If changes were made to your branch [e.g. in GitLab] by someone else, and then you want to keep working on the branch:
Changed:
<
<
  • git fetch --all # (you must do this in order to get access to remote changes in Panoptes/master or any other branch)
>
>
  • git fetch --all # (you must do this in order to get access to remote changes in Panoptes/2018-patches or any other branch)
 
  • git lb-checkout Panoptes/asolomin-MirrAlign-20170123 Rich/RichMirrorAlignmentOnline
Changed:
<
<
After the merge requests are applied and only after, when you are ready to start work again you will want to update your local packages from the master, in case there were any changes in the master in the meantime:
>
>
After the merge requests are applied and only after, when you are ready to start work again you will want to update your local packages from the 2018-patches, in case there were any changes in the 2018-patches in the meantime:
 
  • cd $User_release_area/PanoptesDev_v7r3
Changed:
<
<
  • git fetch --all # (Remember, you must do this in order to get access to remote changes in Panoptes/master or any other branch)
  • git lb-checkout Panoptes/master Rich/RichMirrorAlignmentOnline (and the same for ALL other packages you checked out from master, unless you are still working on a branch of a different package)
>
>
  • git fetch --all # (Remember, you must do this in order to get access to remote changes in Panoptes/2018-patches or any other branch)
  • git lb-checkout Panoptes/2018-patches Rich/RichMirrorAlignmentOnline (and the same for ALL other packages you checked out from 2018-patches, unless you are still working on a branch of a different package)
  *If someone else has lb-pushed a new feature branch called that-branch to Panoptes, you can bring that branch instead to your local copy with
  • git fetch --all
  • git lb-checkout Panoptes/that-branch Rich/RichMirrorAlignmentOnline
Changed:
<
<
In this manner you can change your local copy of the repository for a package very simply, from that which exists in one branch to another. However you should be typically starting fresh from the Panoptes/master branch before adding new features.
>
>
In this manner you can change your local copy of the repository for a package very simply, from that which exists in one branch to another. However you should be typically starting fresh from the Panoptes/2018-patches branch before adding new features.
  If you want to see what's been happening with respect to the local code interacting with the git repository, at any stage, just go here:
Line: 273 to 281
  If you no longer want the merge request to be accepted (e.g. it was done by mistake) then close the merge request. If the associated branch is no longer in use you may delete it from the Hlt gitlab page.
Added:
>
>

Cherry-picking commits from 2018-patches into master

In the new workflow, we always work with 2018-patches as our starting branch, and we will use 2018-patches as the default branch at the pit as well. However, since for the time being we also want the upgrade code to have all of the changes we made, we are required to manually cherry-pick commits we make to 2018-patches to a new branch, and then merge that new branch into master.

There are several ways to do this, which will not be discussed here. However this is something that you have to do, so if you can't figure it out please contact all the mirror alignment experts for advice.

 

Notes from Paras about our global workflow

Typically before we submit a merge request, we want to "svn update" or whatever the equivalent of getting changes that others made in the repository over to our branch, and then "rebased". However, as of June 2017 I am still not exactly sure how to do it correctly.

Changed:
<
<
Fortunately with GitLab, we can leave (simple) rebasing to the Project managers. Just prepare your change then commit. You can start your merge request in gitlab right away, but if you don't want it to be actually merged please put "WIP: " in front of the title of the merge request. Then you can commit more to that branch if you want. But as soon as it is ready please remove "WIP: " and ask Jordi or Chris to merge the branch. Then after the branch is merged update locally to what is in master:
>
>
Fortunately with GitLab, we can leave (simple) rebasing to the Project managers. Just prepare your change then commit. You can start your merge request in gitlab right away, but if you don't want it to be actually merged please put "WIP: " in front of the title of the merge request. Then you can commit more to that branch if you want. But as soon as it is ready please remove "WIP: " and ask Jordi or Chris to merge the branch. Then after the branch is merged update locally to what is in 2018-patches:
 
Changed:
<
<
  • git fetch --all # (you must do this in order to get access to remote changes in Panoptes/master or any other branch)
  • git lb-checkout Panoptes/master Rich/RichMirrAlign (or instead of Rich/RichMirrAlign, whichever package you are going to work on)
>
>
  • git fetch --all # (you must do this in order to get access to remote changes in Panoptes/2018-patches or any other branch)
  • git lb-checkout Panoptes/2018-patches Rich/RichMirrAlign (or instead of Rich/RichMirrAlign, whichever package you are going to work on)
 

Further details on git commands (thanks to Jordi)

Changed:
<
<
1. You should not associate the master branch with the SVN trunk. In SVN, tagged packages were expected to work, and committers could add unstable stuff to the trunk. With the new workflow that we believe is reasonable for Panoptes, the master branch is expected to always be stable, as opposed to the SVN trunk. Unstable contributions to git should take place in branches. So, to summarize, in svn, new contributions go from an unstable trunk to stable branches and from there to release. In git, new contributions go from unstable branches to stable master and from there to release.
>
>
1. You should not associate the 2018-patches / master branch with the SVN trunk. In SVN, tagged packages were expected to work, and committers could add unstable stuff to the trunk. With the new workflow that we believe is reasonable for Panoptes, the 2018-patches / master branch is expected to always be stable, as opposed to the SVN trunk. Unstable contributions to git should take place in branches. So, to summarize, in svn, new contributions go from an unstable trunk to stable branches and from there to release. In git, new contributions go from unstable branches to stable 2018-patches / master and from there to release.
  2. Basically the only place a package v-tag should show up is within the CMakeLists and cmt/requirements of each package. Otherwise we do not tag packages officially (they can be tagged unofficially in commits and in the release notes). However, the Panoptes project should be the only thing that is officially v-tagged.

Revision 312017-10-28 - AnatolySolomin

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 243 to 243
 
  • cd $User_release_area/PanoptesDev_v7r3
  • make install
  • cd /afs/cern.ch/user/a/asolomin/public/rich_align_test (or wherever your test is)
Changed:
<
<
>
>
 

Slight differences to the above for the Hlt project

Revision 302017-10-26 - AnatolySolomin

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 101 to 101
 Only once and forever do these three things:
  • Follow Prerequisites
  • git config --global user.name "Anatoly Solomin" (where Anatoly Solomin = your name)
Changed:
<
<
>
>
  These should be done every time you want to start coding (NOTE the choice of User_release_area is completely up to you, actually, but should be in your AFS area):
  • mkdir /afs/cern.ch/user/a/asolomin/gituser (where asolomin = your username)
Line: 109 to 109
 
  • LbLogin (for some reason you have to run this after setting the user release area)
  • cd $User_release_area
Changed:
<
<
Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here we assume the newest is v7r2, you should check in gitlab):
  • lb-dev --nightly lhcb-head Panoptes/v7r2
    • This creates a local git repository in PanoptesDev_v7r2. This is called a "satellite project".
    • (NOTE we do need to use the nightlies, unless we know that other people are working on code that may affect us... simply use lb-dev Panoptes/v7r2 instead if this is not the immediate case)
>
>
Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here we assume the newest is v7r3, you should check in gitlab):
  • lb-dev --nightly lhcb-head Panoptes/v7r3
    • This creates a local git repository in PanoptesDev_v7r3. This is called a "satellite project".
    • (NOTE we do need to use the nightlies, unless we know that other people are working on code that may affect us... simply use lb-dev Panoptes/v7r3 instead if this is not the immediate case)
  These you do as long as you are working with the project and version in the previous line (NOTE the "Dev"):
Changed:
<
<
  • cd $User_release_area/PanoptesDev_v7r2
>
>
  • cd $User_release_area/PanoptesDev_v7r3
 
  • git lb-use Panoptes
    • This adds the gitlab remote to the local repository of the satellite project, and fetches that remote (makes a local copy of the remote repository, although you still don't see it in your working tree).
Line: 125 to 125
  * [new tag] v1r0 -> Panoptes/v1r0
[...]
* [new tag] v7r1 -> Panoptes/v7r1
Changed:
<
<
* [new tag] v7r2 -> Panoptes/v7r2
>
>
* [new tag] v7r3 -> Panoptes/v7r3
 

You will see that there are several references that you can use.

Line: 212 to 212
 
  • git lb-checkout Panoptes/asolomin-MirrAlign-20170123 Rich/RichMirrorAlignmentOnline

After the merge requests are applied and only after, when you are ready to start work again you will want to update your local packages from the master, in case there were any changes in the master in the meantime:

Changed:
<
<
  • cd $User_release_area/PanoptesDev_v7r2
>
>
  • cd $User_release_area/PanoptesDev_v7r3
 
  • git fetch --all # (Remember, you must do this in order to get access to remote changes in Panoptes/master or any other branch)
  • git lb-checkout Panoptes/master Rich/RichMirrorAlignmentOnline (and the same for ALL other packages you checked out from master, unless you are still working on a branch of a different package)
Line: 240 to 240
 new shell ( e.g. ssh -Y asolomin@lxplusNOSPAMPLEASE.cern.ch )
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser
  • LbLogin
Changed:
<
<
  • cd $User_release_area/PanoptesDev_v7r2
>
>
  • cd $User_release_area/PanoptesDev_v7r3
 
  • make install
  • cd /afs/cern.ch/user/a/asolomin/public/rich_align_test (or wherever your test is)
Changed:
<
<
>
>
 

Slight differences to the above for the Hlt project

Revision 292017-07-24 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 18 to 18
  Please make your own branch of any code you change. Then simply lb-checkout the code from your branch into the AlignmentRelease area when you want to install it.
Added:
>
>
Read the rest of this TWiki first, but more information about the code and how it exists at the pit can be found here: LHCbRichMirrorAlignCodeOnline
 

git

It is time to git! Please read everything and please follow the workflow Anatoly describes. If you have any questions email the alignment-development mailing list. If we cannot answer your question, email Jordi, and if all else fails then Marco Clemencic / lhcb-core-soft@cernNOSPAMPLEASE.ch

Line: 78 to 80
 

Anatoly's example git workflow for Panoptes [or Rec]

Changed:
<
<
Anatoly was kind enough to provide us with his git workflow so far. He is still learning as are we, but this will certainly help you get started. Paras is keeping this updated as new information arrives.
>
>
Anatoly was kind enough to provide us with his git workflow so far (and thanks to Jordi for help). He is still learning as are we, but this will certainly help you get started.

Paras is keeping this updated as new information arrives.

NOTE that this is an example, your use case may be different!

 

To build

Line: 89 to 95
 Create a new shell
Changed:
<
<
Check your git version
  • git version (sadly there are version-dependent commands below)
>
>
Know your git version
  • git version (there may be version-dependent commands below)
  Only once and forever do these three things:
Line: 103 to 109
 
  • LbLogin (for some reason you have to run this after setting the user release area)
  • cd $User_release_area
Changed:
<
<
Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here I assume the newest is v7r2, you should check in gitlab):
>
>
Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here we assume the newest is v7r2, you should check in gitlab):
 
  • lb-dev --nightly lhcb-head Panoptes/v7r2
Added:
>
>
    • This creates a local git repository in PanoptesDev_v7r2. This is called a "satellite project".
    • (NOTE we do need to use the nightlies, unless we know that other people are working on code that may affect us... simply use lb-dev Panoptes/v7r2 instead if this is not the immediate case)
  These you do as long as you are working with the project and version in the previous line (NOTE the "Dev"):
  • cd $User_release_area/PanoptesDev_v7r2
  • git lb-use Panoptes
Added:
>
>
    • This adds the gitlab remote to the local repository of the satellite project, and fetches that remote (makes a local copy of the remote repository, although you still don't see it in your working tree).
 
Changed:
<
<
To checkout a package (e.g before you start work on a new package for the first time)
  • git fetch --all
  • git lb-checkout Panoptes/master Rich/RichMirrAlign (or perhaps a different package)
>
>
The output of this last command will give you information like this:
 * [new branch]      master     -> Panoptes/master <br>
 * [new tag]         v1r0       -> Panoptes/v1r0 <br>
[...] <br>
 * [new tag]         v7r1       -> Panoptes/v7r1 <br>
 * [new tag]         v7r2       -> Panoptes/v7r2 
 
Changed:
<
<
Now you can edit the code. You would obviously not necessarily edit the same files, and of course you should use your editor of choice (e.g. nano, emacs, vi, etc...) or nano as in this example:
  • =nano Rich/RichMirrAlign/CMakeLists.txt =
  • =nano Rich/RichMirrAlign/cmt/requirements =
  • nano Rich/RichMirrAlign/doc/release_notes.html
    • Always update the release notes!, only change the other two if the package "version" needs to be updated
>
>
You will see that there are several references that you can use. Some of them are branches (in this example, only master) and some of them are tags. You will refer to them as Panoptes/[reference], as in the last column of the previous output.

Let's say you want to develop on top of the Rich/RichMirrorAlignmentOnline package that you can find on the master branch. To checkout a package (e.g before you start work on a new package for the first time):

  • git fetch --all (ALWAYS do this before any lb-checkout, it updates the local information about branches with the current information from the git repository.)
  • git lb-checkout Panoptes/master Rich/RichMirrorAlignmentOnline (or perhaps a different package that you might be working on (e.g. Rich/RichMirrAlign))
    • This is the git equivalent of checking out the package from the SVN head.

Now you may edit the code and add new features. You would obviously not necessarily edit the same files as shown in this example, but whatever pieces of code you actually wanted to edit.

Of course you should use your editor of choice (e.g. emacs, vi, etc...) or nano as in this example:

  • nano Rich/RichMirrorAlignmentOnline/CMakeLists.txt
  • nano Rich/RichMirrorAlignmentOnline/cmt/requirements
  • nano Rich/RichMirrorAlignmentOnline/doc/release.notes
    • If you make any code change, always update the release notes! (only change the other two files if the package "version" needs to be updated)
 
Changed:
<
<
Check that your code compiles:
>
>
Check that your code compiles, particularly if you made a change to C++ code:
 
  • make -j 8 (use 8 cores)

Get a list of files that changed:

Changed:
<
<
  • git status or
>
>
  • git status
    • lets you know which files have changed since your last git lb-push ...
 
Changed:
<
<
Check all of your changes (except whitespace changes)
>
>
Or better yet, check all of your changes:
  • git diff
    • shows all changes between what you checked out and your last commit.
<!--   
 
  • git diff --ignore-blank-lines --ignore-space-change (only works in git version 1.8.4 or higher)
  • git diff --ignore-space-change (always works)
Added:
>
>
-->
  For everything that changed and any new files use git add (e.g. for the previous example):
Changed:
<
<
  • git add Rich/RichMirrAlign/src/RichMirrAlign.cpp
  • git add Rich/RichMirrAlign/src/RichMirrAlignFcn.cpp
  • git add Rich/RichMirrAlign/src/RichMirrAlignFcn.h
  • git add Rich/RichMirrAlign/doc/release_notes.html (Please always update the release notes!!!)
  • git add Rich/RichMirrAlign/CMakeLists.txt
  • git add Rich/RichMirrAlign/cmt/requirements

Then make sure to update the commit message:

  • git commit -m 'RICH-9999 regularizationMode introduced.'
RICH-9999 is the associated JIRA task, which automatically links the commit (or later, merge request) to the JIRA task
>
>
  • git add Rich/RichMirrorAlignmentOnline/doc/release.notes (Please always update the release notes!!!)
  • git add Rich/RichMirrorAlignmentOnline/CMakeLists.txt
  • git add Rich/RichMirrorAlignmentOnline/cmt/requirements

Then make sure to update the commit message (for example):

  • git commit -m 'RICH-9999 Updated Release Notes.'
RICH-9999 is the associated JIRA task (if one exists for the task you are working on), which automatically links the commit (or later, merge request) to the JIRA task

Note, there is a shortcut for the above two sets of commands:

  • git commit -a -m 'RICH-9999 Updated Release Notes.'
    • is a combination of git add for all files that changed (but only those that already existed, new files you must add manually) and git commit
 
Changed:
<
<
If after your commit, you edit a file, say Rich/RichMirrAlign/doc/release_notes.html you must git add it again:
  • git add Rich/RichMirrAlign/doc/release_notes.html
>
>
If after your commit, you edit a file, say Rich/RichMirrorAlignmentOnline/doc/release.notes you must git add it again:
  • git add Rich/RichMirrorAlignmentOnline/doc/release.notes
  If after your commit, whether you changed any files or not, you wish to edit or expand your commit description before a push use:
  • git commit --amend
Line: 152 to 184
 You should try to make a new commit every time you've made modifications that can be considered a single unit of changes.

You will now give your branch a name. You could choose anything (but pick something unique!). However let's try to stick to the following prescription:

Changed:
<
<
username-YYYYMMDD-title (e.g. asolomin-20170123-RICHMirrorAdjust).
So for example, to push your changes to a new branch called asolomin-20170123-RICHMirrorAdjust:
  • git lb-push Panoptes asolomin-20170123-RICHMirrorAdjust
>
>
username-title-YYYYMMDD (e.g. asolomin-MirrAlign-20170123).
So for example, to push your changes to a new branch called asolomin-MirrAlign-20170123:
  • git lb-push Panoptes asolomin-MirrAlign-20170123

Here, your equivalent of username-title-YYYYMMDD is the name of your development branch, and you will be able to see it in gitlab.

 
Changed:
<
<
Then, submit a merge request for asolomin-20170123-RICHMirrorAdjust from the Panoptes gitlab page
>
>
If you then would like the commits in username-title-YYYYMMDD to be merged into master, you must perform a Gitlab merge request. You may submit a merge request for asolomin-MirrAlign-20170123 from the Panoptes gitlab page
 You can also use the hyperlink provided when you commit, though if it's a sensitive change you may want to look at your commit in gitlab first.
Changed:
<
<
NOTE: Your pushed branch (that contains your commit) MUST COMPILE locally to get accepted (though it may not compile against the nightlies).
>
>
NOTE: Your pushed branch (that contains your commit) MUST COMPILE locally to get accepted (though it may not compile against the nightlies, consult Chris and Jordi if you find this problem).
  Important: if you noticed a mistake, fix it, and immediately want to commit a corrected version to a new branch, you should
Changed:
<
<
  • git lb-push Panoptes asolomin-20170123-RICHMirrorAdjust2 (i.e. any different name)
Then, submit a merge request for asolomin-20170123-RICHMirrorAdjust2, instead of asolomin-20170123-RICHMirrorAdjust
>
>
  • git lb-push Panoptes asolomin-MirrAlign-20170123-2 (i.e. any different name)
Then, submit a merge request for asolomin-MirrAlign-20170123-2, instead of asolomin-MirrAlign-20170123 Go ahead and delete the branch.
  Now say you already made a merge request, then you could commit a fix and push it to the same branch for which you made the merge request. Before you start work on the fix however, make sure to change the first word of the title of your merge request to "WIP: " (work-in-progress). This instructs the Project manager to wait until you remove "WIP: " to merge the branch.

If you no longer want the merge request to be accepted (e.g. it was done by mistake) then close the merge request. If the associated branch is no longer in use you may delete it from the Panoptes gitlab page.

Changed:
<
<
To continue working on the same branch after a git lb-push (presuming no other changes were made to your branch [e.g. in GitLab]):
>
>
To continue working on the same branch after a git lb-push you don't need to do anything, just change code, commit, and push, and repeat the cycle until ready for a merge request or until your branch is merged. If you already opened a merge request for a branch, you can still keep pushing to that same branch and it will become part of the existing merge request as well.

This shouldn't happen in our workflow, but just so you know about it... If changes were made to your branch [e.g. in GitLab] by someone else, and then you want to keep working on the branch:

 
  • git fetch --all # (you must do this in order to get access to remote changes in Panoptes/master or any other branch)
Changed:
<
<
  • git lb-checkout Panoptes/asolomin-20170123-RICHMirrorAdjust Rich/RichMirrAlign
However if you push again to the same branch then it will become part of any existing merge request as well. You can always keep pushing changes to new branches, this is usually preferred unless someone needs you to fix your merge request.
>
>
  • git lb-checkout Panoptes/asolomin-MirrAlign-20170123 Rich/RichMirrorAlignmentOnline
 
Changed:
<
<
After the merge requests are applied and only after, when you are ready to start work again you will want to update your local packages from the master, incase there were any changes in the master in the meantime
>
>
After the merge requests are applied and only after, when you are ready to start work again you will want to update your local packages from the master, in case there were any changes in the master in the meantime:
 
  • cd $User_release_area/PanoptesDev_v7r2
Changed:
<
<
  • git fetch --all # (you must do this in order to get access to remote changes in Panoptes/master or any other branch)
  • git lb-checkout Panoptes/master Rich/RichMirrAlign
>
>
  • git fetch --all # (Remember, you must do this in order to get access to remote changes in Panoptes/master or any other branch)
  • git lb-checkout Panoptes/master Rich/RichMirrorAlignmentOnline (and the same for ALL other packages you checked out from master, unless you are still working on a branch of a different package)

*If someone else has lb-pushed a new feature branch called that-branch to Panoptes, you can bring that branch instead to your local copy with

  • git fetch --all
  • git lb-checkout Panoptes/that-branch Rich/RichMirrorAlignmentOnline
In this manner you can change your local copy of the repository for a package very simply, from that which exists in one branch to another. However you should be typically starting fresh from the Panoptes/master branch before adding new features.

If you want to see what's been happening with respect to the local code interacting with the git repository, at any stage, just go here:

git log

  • tracks everything you have been doing

If you want it with diffs:

git log -U3 (3 can be changed to any number)

  • git log with diffs from 3 (or another number) lines before to 3 (or another number) lines after any changes

Please also read the further notes on the workflow.

 

To run

Added:
>
>
Say you wanted to run a test of your code. Right now we can only test Rich/RichMirrorAlignmentOnline on the farm. But if you edited Rich/RichMirrAlign you could test it in the following manner:
 new shell ( e.g. ssh -Y asolomin@lxplusNOSPAMPLEASE.cern.ch )
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser
  • LbLogin
Line: 217 to 273
  If you no longer want the merge request to be accepted (e.g. it was done by mistake) then close the merge request. If the associated branch is no longer in use you may delete it from the Hlt gitlab page.
Added:
>
>
 

Notes from Paras about our global workflow

Typically before we submit a merge request, we want to "svn update" or whatever the equivalent of getting changes that others made in the repository over to our branch, and then "rebased". However, as of June 2017 I am still not exactly sure how to do it correctly.

Line: 230 to 287
  1. You should not associate the master branch with the SVN trunk. In SVN, tagged packages were expected to work, and committers could add unstable stuff to the trunk. With the new workflow that we believe is reasonable for Panoptes, the master branch is expected to always be stable, as opposed to the SVN trunk. Unstable contributions to git should take place in branches. So, to summarize, in svn, new contributions go from an unstable trunk to stable branches and from there to release. In git, new contributions go from unstable branches to stable master and from there to release.
Changed:
<
<
2. Simple git workflow for Panoptes (you should of course pick the newest tagged Panoptes version):

lb-dev Panoptes v7r2

This creates a local git repository in PanoptesDev_v7r2. This is called a "satellite project". Then,

cd PanoptesDev_v7r2
git lb-use Panoptes

This adds the gitlab remote to the local repository of the satellite project, and fetches that remote (makes a local copy of the remote repository, although you still don't see it in your working tree).

The output of this last command will give you information like this:

* [new branch] master -> Panoptes/master
* [new tag] v1r0 -> Panoptes/v1r0
[...]
* [new tag] v6r0 -> Panoptes/v5r5
* [new tag] v7r1 -> Panoptes/v7r1

You will see that there are several references that you can use. Some of them are branches (right now only master) and some of them are tags. You will refer to them as Panoptes/[reference], as in the last column of the previous output.

Lets say you want to develop on top of the Rich/RichMirrorAlignmentOnline package that you can find on the master branch. You would do

git fetch --all
git lb-checkout Panoptes/master Rich/RichMirrorAlignmentOnline

This is the equivalent of checking out the package from the head. Then you would add new features to the mirror alignment software, and maybe you will commit unfinished work that you want to share with others. In this case, you would of course make the edits and then git add them (git add must be done immediately before a git commit / git lb-push; if any file changes git add must be done again, and use git commit --amend if you want to edit the git commit -m note):

... edit filename ...

git add filename
git commit -m "edited filename"

and then lb-push to a new branch:

git lb-push Panoptes newfeature

Here, newfeature is the name of your development branch, and you will be able to see it in gitlab. It should be short, but descriptive. If you then would like the commits in newfeature to be merged into master you must perform a Gitlab merge request (there should be a weblink given to you when you lb-push).

If someone else has lb-pushed a new feature branch called that-branch to Panoptes, you can bring it to your local copy with

git fetch --all
git lb-checkout Panoptes/that-branch Rich/RichMirrorAlignmentOnline

You can also tag specific commits on the branch, so you can easily refer to them with others. Please don't use v-tags: the Panoptes project should be the only thing that is v-tagged. If v-tags help you understand better where you are, please prefix them with some short context, e.g. align-v1r0. Basically the only place a package v-tag should show up is within the CMakeLists and cmt/requirements of each package

>
>
2. Basically the only place a package v-tag should show up is within the CMakeLists and cmt/requirements of each package. Otherwise we do not tag packages officially (they can be tagged unofficially in commits and in the release notes). However, the Panoptes project should be the only thing that is officially v-tagged.
  3. The lb-use and lb-checkout commands also solve the case where you want to use packages as plugins from other projects. For example, you can very consistently do:
Line: 295 to 301
  5. Currently Jordi Garra Tico and Chris Jones are the only people that can approve merge requests.
Deleted:
<
<

List of questions to be asked and answered (by Jordi, etc...) at an appropriate time

Is there a way, say after you commit “mirralign23”, to do something similar to “svn update” to get your local branch to be in sync with the master branch (say, if your changes are accepted, then someone else changes it), or would you have to check out the head version each time?

  • Q: You cannot "git lb-push" to a branch already existing in the remote repository, unless you checked out from that specific branch?
    • A (?) : I think, it is without "unless", meaning that such a branch is always unique, but keep this Q too.
 

Frequently used commands

Changed:
<
<
see above for starters, but here are a few more

git diff

  • shows all changes between what you checked out and your last commit.

git status

  • lets you know which files have changed since your last git lb-push

git commit -a -m "Tagged Rich/RichMirrorAlignmentOnline as v68r43. Updated Release Notes."

  • is a combination of git add for all files that changed (it does not cover new files to be added to the repository though!) and git commit -m "Hello"

git log

  • tracks everything you have been doing

git log -U3 (3 can be changed to any number)

  • git log with diffs from 3 (or another number) lines before to 3 (or another number) lines after any changes
>
>
If there are any we forgot above, we should place them here.
 

Troubleshooting

Revision 282017-07-06 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 103 to 103
 
  • LbLogin (for some reason you have to run this after setting the user release area)
  • cd $User_release_area
Changed:
<
<
Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here I assume the newest is v7r1, you should check in gitlab):
  • lb-dev --nightly lhcb-head Panoptes/v7r1
>
>
Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here I assume the newest is v7r2, you should check in gitlab):
  • lb-dev --nightly lhcb-head Panoptes/v7r2
  These you do as long as you are working with the project and version in the previous line (NOTE the "Dev"):
Changed:
<
<
  • cd $User_release_area/PanoptesDev_v7r1
>
>
  • cd $User_release_area/PanoptesDev_v7r2
 
  • git lb-use Panoptes

To checkout a package (e.g before you start work on a new package for the first time)

Line: 175 to 175
 You can always keep pushing changes to new branches, this is usually preferred unless someone needs you to fix your merge request.

After the merge requests are applied and only after, when you are ready to start work again you will want to update your local packages from the master, incase there were any changes in the master in the meantime

Changed:
<
<
  • cd $User_release_area/PanoptesDev_v7r1
>
>
  • cd $User_release_area/PanoptesDev_v7r2
 
  • git fetch --all # (you must do this in order to get access to remote changes in Panoptes/master or any other branch)
  • git lb-checkout Panoptes/master Rich/RichMirrAlign
Line: 184 to 184
 new shell ( e.g. ssh -Y asolomin@lxplusNOSPAMPLEASE.cern.ch )
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser
  • LbLogin
Changed:
<
<
  • cd $User_release_area/PanoptesDev_v7r1
>
>
  • cd $User_release_area/PanoptesDev_v7r2
 
  • make install
  • cd /afs/cern.ch/user/a/asolomin/public/rich_align_test (or wherever your test is)
Changed:
<
<
>
>
 

Slight differences to the above for the Hlt project

Line: 232 to 232
  2. Simple git workflow for Panoptes (you should of course pick the newest tagged Panoptes version):
Changed:
<
<
lb-dev Panoptes v7r1
>
>
lb-dev Panoptes v7r2
 
Changed:
<
<
This creates a local git repository in PanoptesDev_v7r1. This is called a "satellite project". Then,
>
>
This creates a local git repository in PanoptesDev_v7r2. This is called a "satellite project". Then,
 
Changed:
<
<
cd PanoptesDev_v7r1
>
>
cd PanoptesDev_v7r2
 git lb-use Panoptes

This adds the gitlab remote to the local repository of the satellite project,

Revision 272017-07-03 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Changed:
<
<

git instructions for the RICH mirror alignment

>
>
-- ParasNaik - 2016-04-12

git instructions for the RICH mirror alignment

  Because we are no longer using SVN.
Line: 10 to 12
 
Added:
>
>

Nota Bene

NEVER edit the code directly at the pit (e.g. in the AlignmentRelease area). When using multiple projects with git, things can get extremely confused and it will be very difficult to find and merge any changes into to code!

Please make your own branch of any code you change. Then simply lb-checkout the code from your branch into the AlignmentRelease area when you want to install it.

 

git

It is time to git! Please read everything and please follow the workflow Anatoly describes. If you have any questions email the alignment-development mailing list. If we cannot answer your question, email Jordi, and if all else fails then Marco Clemencic / lhcb-core-soft@cernNOSPAMPLEASE.ch

Line: 332 to 340
  Jordi will update this repository just once in a while but not very frequently.

Jordi has created the gitlab group lhcb-rich, intended for rich related repositories. https://gitlab.cern.ch/groups/lhcb-rich

Deleted:
<
<

The End

-- ParasNaik - 2016-04-12

 \ No newline at end of file

Revision 262017-06-14 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"

Revision 252017-06-12 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 209 to 209
  If you no longer want the merge request to be accepted (e.g. it was done by mistake) then close the merge request. If the associated branch is no longer in use you may delete it from the Hlt gitlab page.
Added:
>
>

Notes from Paras about our global workflow

Typically before we submit a merge request, we want to "svn update" or whatever the equivalent of getting changes that others made in the repository over to our branch, and then "rebased". However, as of June 2017 I am still not exactly sure how to do it correctly.

Fortunately with GitLab, we can leave (simple) rebasing to the Project managers. Just prepare your change then commit. You can start your merge request in gitlab right away, but if you don't want it to be actually merged please put "WIP: " in front of the title of the merge request. Then you can commit more to that branch if you want. But as soon as it is ready please remove "WIP: " and ask Jordi or Chris to merge the branch. Then after the branch is merged update locally to what is in master:

  • git fetch --all # (you must do this in order to get access to remote changes in Panoptes/master or any other branch)
  • git lb-checkout Panoptes/master Rich/RichMirrAlign (or instead of Rich/RichMirrAlign, whichever package you are going to work on)
 

Further details on git commands (thanks to Jordi)

1. You should not associate the master branch with the SVN trunk. In SVN, tagged packages were expected to work, and committers could add unstable stuff to the trunk. With the new workflow that we believe is reasonable for Panoptes, the master branch is expected to always be stable, as opposed to the SVN trunk. Unstable contributions to git should take place in branches. So, to summarize, in svn, new contributions go from an unstable trunk to stable branches and from there to release. In git, new contributions go from unstable branches to stable master and from there to release.

Line: 240 to 249
  Lets say you want to develop on top of the Rich/RichMirrorAlignmentOnline package that you can find on the master branch. You would do
Changed:
<
<
git fetch --all
>
>
git fetch --all
 git lb-checkout Panoptes/master Rich/RichMirrorAlignmentOnline

This is the equivalent of checking out the package from the head.

Line: 250 to 259
  ... edit filename ...
Changed:
<
<
git add =filename
>
>
git add filename
 git commit -m "edited filename"

and then lb-push to a new branch:

Line: 269 to 278
  3. The lb-use and lb-checkout commands also solve the case where you want to use packages as plugins from other projects. For example, you can very consistently do:

git lb-use Rec

Changed:
<
<
git fetch --all
>
>
git fetch --all
  git lb-checkout Rec/v20r1 Rich/RichAlignment

So this is how you would use the Phys/Rec package in Panoptes.

Line: 295 to 304
 git status
  • lets you know which files have changed since your last git lb-push
Changed:
<
<
=git commit -a -m "Hello"
>
>
git commit -a -m "Tagged Rich/RichMirrorAlignmentOnline as v68r43. Updated Release Notes."
 
  • is a combination of git add for all files that changed (it does not cover new files to be added to the repository though!) and git commit -m "Hello"

git log

Revision 242017-05-22 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"

Revision 232017-04-20 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 280 to 280
 

List of questions to be asked and answered (by Jordi, etc...) at an appropriate time

Changed:
<
<
<!-- What is the reason to commit (but without a push) the fact that a package was checked out? [*NOTE* No one has ever complained that we didn't...] -->

Also, is there a way, say after you commit “mirralign23”, to do something similar to “svn update” to get your local branch to be in sync with the master branch (say, if your changes are accepted, then someone else changes it), or would you have to check out the head version each time?

>
>
Is there a way, say after you commit “mirralign23”, to do something similar to “svn update” to get your local branch to be in sync with the master branch (say, if your changes are accepted, then someone else changes it), or would you have to check out the head version each time?
 
  • Q: You cannot "git lb-push" to a branch already existing in the remote repository, unless you checked out from that specific branch?
    • A (?) : I think, it is without "unless", meaning that such a branch is always unique, but keep this Q too.
Line: 291 to 289
  see above for starters, but here are a few more
Added:
>
>
git diff
  • shows all changes between what you checked out and your last commit.
 git status
  • lets you know which files have changed since your last git lb-push
Changed:
<
<
git diff
  • shows all changes between what you checked out and your last-commit.
>
>
=git commit -a -m "Hello"
  • is a combination of git add for all files that changed (it does not cover new files to be added to the repository though!) and git commit -m "Hello"
  git log
  • tracks everything you have been doing

Revision 222017-04-18 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"

Revision 212017-04-10 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 92 to 92
 These should be done every time you want to start coding (NOTE the choice of User_release_area is completely up to you, actually, but should be in your AFS area):
  • mkdir /afs/cern.ch/user/a/asolomin/gituser (where asolomin = your username)
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser (where asolomin = your username)
Added:
>
>
  • LbLogin (for some reason you have to run this after setting the user release area)
 
  • cd $User_release_area

Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here I assume the newest is v7r1, you should check in gitlab):

Line: 174 to 175
  new shell ( e.g. ssh -Y asolomin@lxplusNOSPAMPLEASE.cern.ch )
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser
Added:
>
>
  • LbLogin
 
  • cd $User_release_area/PanoptesDev_v7r1
  • make install
  • cd /afs/cern.ch/user/a/asolomin/public/rich_align_test (or wherever your test is)

Revision 202017-04-08 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 106 to 106
 
  • git lb-checkout Panoptes/master Rich/RichMirrAlign (or perhaps a different package)

Now you can edit the code. You would obviously not necessarily edit the same files, and of course you should use your editor of choice (e.g. nano, emacs, vi, etc...) or nano as in this example:

Changed:
<
<
  • nano Rich/RichMirrAlign/src/RichMirrAlign* &
  • nano Rich/RichMirrAlign/doc/release_notes.html Rich/RichMirrAlign/CMakeLists.txt Rich/RichMirrAlign/cmt/requirements & (always update the release notes!, only change the other two if the package "version" needs to be updated)
>
>
  • =nano Rich/RichMirrAlign/CMakeLists.txt =
  • =nano Rich/RichMirrAlign/cmt/requirements =
  • nano Rich/RichMirrAlign/doc/release_notes.html
    • Always update the release notes!, only change the other two if the package "version" needs to be updated
  Check that your code compiles:
  • make -j 8 (use 8 cores)
Line: 131 to 133
 
  • git commit -m 'RICH-9999 regularizationMode introduced.'
RICH-9999 is the associated JIRA task, which automatically links the commit (or later, merge request) to the JIRA task
Changed:
<
<
If after your first commit, you want to edit or expand your commit description before a push use:
>
>
If after your commit, you edit a file, say Rich/RichMirrAlign/doc/release_notes.html you must git add it again:
  • git add Rich/RichMirrAlign/doc/release_notes.html

If after your commit, whether you changed any files or not, you wish to edit or expand your commit description before a push use:

 
  • git commit --amend

As new commits are only stored in your local repository, there's no cost to committing often.

Revision 192017-04-06 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 100 to 100
 These you do as long as you are working with the project and version in the previous line (NOTE the "Dev"):
  • cd $User_release_area/PanoptesDev_v7r1
  • git lb-use Panoptes
Deleted:
<
<
  • git fetch --all
  To checkout a package (e.g before you start work on a new package for the first time)
Added:
>
>
  • git fetch --all
 
  • git lb-checkout Panoptes/master Rich/RichMirrAlign (or perhaps a different package)

Now you can edit the code. You would obviously not necessarily edit the same files, and of course you should use your editor of choice (e.g. nano, emacs, vi, etc...) or nano as in this example:

Line: 155 to 155
 If you no longer want the merge request to be accepted (e.g. it was done by mistake) then close the merge request. If the associated branch is no longer in use you may delete it from the Panoptes gitlab page.

To continue working on the same branch after a git lb-push (presuming no other changes were made to your branch [e.g. in GitLab]):

Added:
>
>
  • git fetch --all # (you must do this in order to get access to remote changes in Panoptes/master or any other branch)
 
  • git lb-checkout Panoptes/asolomin-20170123-RICHMirrorAdjust Rich/RichMirrAlign
However if you push again to the same branch then it will become part of any existing merge request as well. You can always keep pushing changes to new branches, this is usually preferred unless someone needs you to fix your merge request.

After the merge requests are applied and only after, when you are ready to start work again you will want to update your local packages from the master, incase there were any changes in the master in the meantime

  • cd $User_release_area/PanoptesDev_v7r1
Changed:
<
<
  • git fetch --all (you must do this in order to get access to remote changes in Panoptes/master or any other branch)
>
>
  • git fetch --all # (you must do this in order to get access to remote changes in Panoptes/master or any other branch)
 
  • git lb-checkout Panoptes/master Rich/RichMirrAlign

To run

Line: 179 to 180
 
  • lb-dev Moore/v25r3 (or latest version, check in gitlab)
  • cd $User_release_area/MooreDev_v25r3
  • git lb-use Hlt
Added:
>
>
  • git fetch --all
 
  • git lb-checkout Hlt/2016-patches Hlt/HltSettings (perhaps you will be asked by HLT Operations to use a different branch, or need to edit a different package)
  • nano Hlt/HltSettings/python/HltSettings/Physics_pp_Draft2016.py (example)
  • nano Hlt/HltSettings/doc/release.notes
Line: 200 to 202
  If you no longer want the merge request to be accepted (e.g. it was done by mistake) then close the merge request. If the associated branch is no longer in use you may delete it from the Hlt gitlab page.
Changed:
<
<

Further Instructions from Jordi, based on Paras's initial questions

>
>

Further details on git commands (thanks to Jordi)

  1. You should not associate the master branch with the SVN trunk. In SVN, tagged packages were expected to work, and committers could add unstable stuff to the trunk. With the new workflow that we believe is reasonable for Panoptes, the master branch is expected to always be stable, as opposed to the SVN trunk. Unstable contributions to git should take place in branches. So, to summarize, in svn, new contributions go from an unstable trunk to stable branches and from there to release. In git, new contributions go from unstable branches to stable master and from there to release.
Changed:
<
<
2. An example git workflow :
>
>
2. Simple git workflow for Panoptes (you should of course pick the newest tagged Panoptes version):
 
Changed:
<
<
lb-dev Panoptes v5r6
>
>
lb-dev Panoptes v7r1
 
Changed:
<
<
This creates a local git repository in PanoptesDev_v5r6. This is called a "satellite project". Then,
>
>
This creates a local git repository in PanoptesDev_v7r1. This is called a "satellite project". Then,
 
Changed:
<
<
cd PanoptesDev_v5r6
>
>
cd PanoptesDev_v7r1
 git lb-use Panoptes
Changed:
<
<
This adds the gitlab remote to the local repository of the satellite project, and fetches that remote (makes a local copy of the remote repository, although you still don't see it in your working tree).
>
>
This adds the gitlab remote to the local repository of the satellite project, and fetches that remote (makes a local copy of the remote repository, although you still don't see it in your working tree).
  The output of this last command will give you information like this:

* [new branch] master -> Panoptes/master
* [new tag] v1r0 -> Panoptes/v1r0
[...]

Changed:
<
<
* [new tag] v5r5 -> Panoptes/v5r5
* [new tag] v5r6 -> Panoptes/v5r6
>
>
* [new tag] v6r0 -> Panoptes/v5r5
* [new tag] v7r1 -> Panoptes/v7r1
 
Changed:
<
<
You will see that there are several references that you can use. Some of them are branches (right now only master) and some of them are tags. You will refer to them as Panoptes/[reference], as in the last column of the previous output.
>
>
You will see that there are several references that you can use. Some of them are branches (right now only master) and some of them are tags. You will refer to them as Panoptes/[reference], as in the last column of the previous output.
  Lets say you want to develop on top of the Rich/RichMirrorAlignmentOnline package that you can find on the master branch. You would do
Added:
>
>
git fetch --all
 git lb-checkout Panoptes/master Rich/RichMirrorAlignmentOnline
Changed:
<
<
This is the equivalent of checking out the package from the head. Then you would add new features to the mirror alignment software, and maybe you will commit unfinished work that you want to share with others. In this case, you would do
>
>
This is the equivalent of checking out the package from the head. Then you would add new features to the mirror alignment software, and maybe you will commit unfinished work that you want to share with others. In this case, you would of course make the edits and then git add them (git add must be done immediately before a git commit=/=git lb-push; if any file changes git add must be done again, and use git commit --amend if you want to edit the git commit -m note):

... edit filename ...

git add =filename git commit -m "edited filename"

and then lb-push to a new branch:

  git lb-push Panoptes newfeature
Changed:
<
<
Here, newfeature is the name of your development branch, and you will be able to see it in gitlab. It should be short, but descriptive. If someone else has lb-pushed a new feature branch to Panoptes, you can bring it to your local copy with
>
>
Here, newfeature is the name of your development branch, and you will be able to see it in gitlab. It should be short, but descriptive. If you then would like the commits in newfeature to be merged into master you must perform a Gitlab merge request (there should be a weblink given to you when you lb-push).

If someone else has lb-pushed a new feature branch called that-branch to Panoptes, you can bring it to your local copy with

 
Changed:
<
<
git fetch Panoptes
>
>
git fetch --all
 git lb-checkout Panoptes/that-branch Rich/RichMirrorAlignmentOnline

You can also tag specific commits on the branch, so you can easily refer to them with others. Please don't use v-tags: the Panoptes project should be the only thing that is v-tagged. If v-tags help you understand better where you are, please prefix them with some short context, e.g. align-v1r0. Basically the only place a package v-tag should show up is within the CMakeLists and cmt/requirements of each package

Changed:
<
<
3. I think the lb-use and lb-checkout commands also solve the case where you want to use packages as plugins from other projects. You can very consistently do
>
>
3. The lb-use and lb-checkout commands also solve the case where you want to use packages as plugins from other projects. For example, you can very consistently do:
 
Changed:
<
<
git lb-use DaVinci
git lb-checkout DaVinci/v38r1 Phys/DaVinci
>
>
git lb-use Rec
git fetch --all git lb-checkout Rec/v20r1 Rich/RichAlignment
 
Changed:
<
<
So this is how you would use the Phys/DaVinci package in Panoptes (regardless of the fact that it would not make any sense).
>
>
So this is how you would use the Phys/Rec package in Panoptes.
 
Changed:
<
<
4. About releasing a new version of Panoptes every time a package has a substantial improvement: I don't have a correct answer to this question. New Panoptes releases are created when there's interest in them. It may be for several reasons: to have new features released, to use a new release of Online, to compile with a new CMTCONFIG,... I never oppose creating a new release if it is requested.
>
>
4. New Panoptes releases are created when there's interest in them. It may be for several reasons: to have new features released, to use a new release of Online, to compile with a new CMTCONFIG,... Jordi never opposes creating a new release if it is requested.
 
Changed:
<
<
5. Who approves merge requests? Currently Jordi Garra Tico and Chris Jones.

6. Git add must be done immediately before a git commit/lb-push; if any file changes git add must be done again.

>
>
5. Currently Jordi Garra Tico and Chris Jones are the only people that can approve merge requests.
 

List of questions to be asked and answered (by Jordi, etc...) at an appropriate time

Changed:
<
<
What is the reason to commit (but without a push) the fact that a package was checked out? [*NOTE* No one has ever complained that we didn't...]
>
>
<!-- What is the reason to commit (but without a push) the fact that a package was checked out? [*NOTE* No one has ever complained that we didn't...] -->
  Also, is there a way, say after you commit “mirralign23”, to do something similar to “svn update” to get your local branch to be in sync with the master branch (say, if your changes are accepted, then someone else changes it), or would you have to check out the head version each time?
Deleted:
<
<
Questions on which Anatoly was not 100% sure:
 
  • Q: You cannot "git lb-push" to a branch already existing in the remote repository, unless you checked out from that specific branch?
Changed:
<
<
    • A: I think, it is without "unless", meaning that such a branch is always unique, but keep this Q too.
  • Q: How during running one could use another project’s package ( before this was possible by getpacking whatever package you want to use, then using e.g. SetupProject Panoptes vXrY --use Rich/RichAlignment ). This is useful, for example, if we work on Rich/RichAlignment but a new version of Rec has not yet been released. Something to figure out later though as hopefully this is a rare case.
    • A: No clue. But of course, our use case is a "multiproject" one (Panoptes + Brunel), and in some situations (e.g. in the Ganga case) we need a recipe for that.
>
>
    • A (?) : I think, it is without "unless", meaning that such a branch is always unique, but keep this Q too.
 

Frequently used commands

Added:
>
>
see above for starters, but here are a few more

git status

  • lets you know which files have changed since your last git lb-push

git diff

  • shows all changes between what you checked out and your last-commit.

git log

  • tracks everything you have been doing

git log -U3 (3 can be changed to any number)

  • git log with diffs from 3 (or another number) lines before to 3 (or another number) lines after any changes
 

Troubleshooting

Q: I pushed a commit to a branch for which I already have a merge request. However my new commits don't show up in that merge request!

Revision 182017-04-05 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 100 to 100
 These you do as long as you are working with the project and version in the previous line (NOTE the "Dev"):
  • cd $User_release_area/PanoptesDev_v7r1
  • git lb-use Panoptes
Added:
>
>
  • git fetch --all
  To checkout a package (e.g before you start work on a new package for the first time)
  • git lb-checkout Panoptes/master Rich/RichMirrAlign (or perhaps a different package)

Revision 172017-04-05 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 94 to 94
 
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser (where asolomin = your username)
  • cd $User_release_area
Changed:
<
<
Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here I assume the newest is v6r0, you should check in gitlab):
  • lb-dev --nightly=lhcb-head Panoptes/v6r0
>
>
Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here I assume the newest is v7r1, you should check in gitlab):
  • lb-dev --nightly lhcb-head Panoptes/v7r1
  These you do as long as you are working with the project and version in the previous line (NOTE the "Dev"):
Changed:
<
<
  • cd $User_release_area/PanoptesDev_v6r0
>
>
  • cd $User_release_area/PanoptesDev_v7r1
 
  • git lb-use Panoptes

To checkout a package (e.g before you start work on a new package for the first time)

Line: 159 to 159
 You can always keep pushing changes to new branches, this is usually preferred unless someone needs you to fix your merge request.

After the merge requests are applied and only after, when you are ready to start work again you will want to update your local packages from the master, incase there were any changes in the master in the meantime

Changed:
<
<
  • cd $User_release_area/PanoptesDev_v6r0
>
>
  • cd $User_release_area/PanoptesDev_v7r1
 
  • git fetch --all (you must do this in order to get access to remote changes in Panoptes/master or any other branch)
  • git lb-checkout Panoptes/master Rich/RichMirrAlign
Line: 167 to 167
  new shell ( e.g. ssh -Y asolomin@lxplusNOSPAMPLEASE.cern.ch )
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser
Changed:
<
<
  • cd $User_release_area/PanoptesDev_v6r0
>
>
  • cd $User_release_area/PanoptesDev_v7r1
 
  • make install
  • cd /afs/cern.ch/user/a/asolomin/public/rich_align_test (or wherever your test is)
Changed:
<
<
>
>
 

Slight differences to the above for the Hlt project

Revision 162017-04-04 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 95 to 95
 
  • cd $User_release_area

Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here I assume the newest is v6r0, you should check in gitlab):

Changed:
<
<
  • lb-dev --nightly=lhcb-head Panoptes v6r0
>
>
  • lb-dev --nightly=lhcb-head Panoptes/v6r0
  These you do as long as you are working with the project and version in the previous line (NOTE the "Dev"):
  • cd $User_release_area/PanoptesDev_v6r0
Line: 175 to 175
 

Slight differences to the above for the Hlt project

Substitute the following above, where appropriate

Changed:
<
<
  • lb-dev Moore v25r3 (or latest version, check in gitlab)
>
>
  • lb-dev Moore/v25r3 (or latest version, check in gitlab)
 
  • cd $User_release_area/MooreDev_v25r3
  • git lb-use Hlt
  • git lb-checkout Hlt/2016-patches Hlt/HltSettings (perhaps you will be asked by HLT Operations to use a different branch, or need to edit a different package)
Line: 268 to 268
 

Troubleshooting

Changed:
<
<
<!-- 
Paras does not know if the section is useful, so he commented it out
>
>
Q: I pushed a commit to a branch for which I already have a merge request. However my new commits don't show up in that merge request!
 
Changed:
<
<
Q: All of my commits have now been merged. I git diff, and the only file that has changed is called .git-lb-checkout ... what should I do?

A: Definitely do not leave it. If that is the only file that is a problem and all of your commits have been merged:

git checkout -- .git-lb-checkout

Actually this may be worth doing in any instance, but we have no idea yet what .git-lb-checkout actually does -->

>
>
A: Push another commit
 

Legacy repository information

Revision 152017-03-24 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 133 to 133
 If after your first commit, you want to edit or expand your commit description before a push use:
  • git commit --amend
Added:
>
>
As new commits are only stored in your local repository, there's no cost to committing often.
You should try to make a new commit every time you've made modifications that can be considered a single unit of changes.
 You will now give your branch a name. You could choose anything (but pick something unique!). However let's try to stick to the following prescription: username-YYYYMMDD-title (e.g. asolomin-20170123-RICHMirrorAdjust).
So for example, to push your changes to a new branch called asolomin-20170123-RICHMirrorAdjust:
Line: 150 to 153
  If you no longer want the merge request to be accepted (e.g. it was done by mistake) then close the merge request. If the associated branch is no longer in use you may delete it from the Panoptes gitlab page.
Added:
>
>
To continue working on the same branch after a git lb-push (presuming no other changes were made to your branch [e.g. in GitLab]):
  • git lb-checkout Panoptes/asolomin-20170123-RICHMirrorAdjust Rich/RichMirrAlign
However if you push again to the same branch then it will become part of any existing merge request as well. You can always keep pushing changes to new branches, this is usually preferred unless someone needs you to fix your merge request.

After the merge requests are applied and only after, when you are ready to start work again you will want to update your local packages from the master, incase there were any changes in the master in the meantime

  • cd $User_release_area/PanoptesDev_v6r0
  • git fetch --all (you must do this in order to get access to remote changes in Panoptes/master or any other branch)
  • git lb-checkout Panoptes/master Rich/RichMirrAlign
 

To run

new shell ( e.g. ssh -Y asolomin@lxplusNOSPAMPLEASE.cern.ch )

Revision 142017-03-24 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 101 to 101
 
  • cd $User_release_area/PanoptesDev_v6r0
  • git lb-use Panoptes
Changed:
<
<
This you do only to checkout a package or update everything you have from master (e.g, do this overtime before you start work on a package and do this every time after our branches have been accepted into the master)
<!--a procedure replicating "svn update" is still being established-->
>
>
To checkout a package (e.g before you start work on a new package for the first time)
 
  • git lb-checkout Panoptes/master Rich/RichMirrAlign (or perhaps a different package)
Changed:
<
<
Now you can edit the code. You would obviously not necessarily edit the same files, and of course you should use your editor of choice (e.g. nano, emacs, vim, etc...) or edit as in this example:
  • edit Rich/RichMirrAlign/src/RichMirrAlign* &
  • edit Rich/RichMirrAlign/doc/release_notes.html Rich/RichMirrAlign/CMakeLists.txt Rich/RichMirrAlign/cmt/requirements & (always update the release notes!, only change the other two if the package "version" needs to be updated)
>
>
Now you can edit the code. You would obviously not necessarily edit the same files, and of course you should use your editor of choice (e.g. nano, emacs, vi, etc...) or nano as in this example:
  • nano Rich/RichMirrAlign/src/RichMirrAlign* &
  • nano Rich/RichMirrAlign/doc/release_notes.html Rich/RichMirrAlign/CMakeLists.txt Rich/RichMirrAlign/cmt/requirements & (always update the release notes!, only change the other two if the package "version" needs to be updated)
  Check that your code compiles:
  • make -j 8 (use 8 cores)

Get a list of files that changed:

  • git status or
Deleted:
<
<
  • git diff --name-only
 
Changed:
<
<
If need be, or to be safe, check all of the changes (except whitespace changes)
>
>
Check all of your changes (except whitespace changes)
 
  • git diff --ignore-blank-lines --ignore-space-change (only works in git version 1.8.4 or higher)
  • git diff --ignore-space-change (always works)
Line: 253 to 251
 
  • Q: How during running one could use another project’s package ( before this was possible by getpacking whatever package you want to use, then using e.g. SetupProject Panoptes vXrY --use Rich/RichAlignment ). This is useful, for example, if we work on Rich/RichAlignment but a new version of Rec has not yet been released. Something to figure out later though as hopefully this is a rare case.
    • A: No clue. But of course, our use case is a "multiproject" one (Panoptes + Brunel), and in some situations (e.g. in the Ganga case) we need a recipe for that.
Added:
>
>

Frequently used commands

 

Troubleshooting

Added:
>
>
<!-- 
Paras does not know if the section is useful, so he commented it out
 Q: All of my commits have now been merged. I git diff, and the only file that has changed is called .git-lb-checkout ... what should I do?

A: Definitely do not leave it. If that is the only file that is a problem and all of your commits have been merged:

Line: 262 to 265
 git checkout -- .git-lb-checkout

Actually this may be worth doing in any instance, but we have no idea yet what .git-lb-checkout actually does

Added:
>
>
-->
 

Legacy repository information

Revision 132017-03-23 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"

git instructions for the RICH mirror alignment

Added:
>
>
Because we are no longer using SVN.
 

Table of Contents

git

Changed:
<
<
We are no longer using SVN. It is time to git! Please read everything and please follow the workflow Anatoly describes. If you have any questions email the alignment-development mailing list. If we cannot answer your question, email Jordi, and if all else fails then Marco Clemencic / lhcb-core-soft@cernNOSPAMPLEASE.ch
>
>
It is time to git! Please read everything and please follow the workflow Anatoly describes. If you have any questions email the alignment-development mailing list. If we cannot answer your question, email Jordi, and if all else fails then Marco Clemencic / lhcb-core-soft@cernNOSPAMPLEASE.ch

The basic idea behind git is that "branches" are used to develop new features of the code, each branch forks from the master branch (the "default" branch) of a git repository, and each new branch is isolated from the other branches, until one chooses to merge them. Typically we use these other branches for development, and then merge them back to the master branch when the new feature is tested and ready. At LHCb we have to ask for a merge request in CERN's GitLab to merge one of our branches into the master branch of a Project. Then the Project manager can decide whether to allow the merge to occur.

 

Standard documentation and other git stuff

Line: 74 to 78
  [ Change projects throughout below, if instead of Panoptes you are working with e.g. Rec ]
Changed:
<
<
Create a new shell ( e.g. ssh   -Y   asolomin@lxplusNOSPAMPLEASE.cern.ch )
>
>
Create a new shell

Check your git version

  • git version (sadly there are version-dependent commands below)
  Only once and forever do these three things:
Changed:
<
<
  • git config --global user.name "Anatoly Solomin"
  • git config --global user.email anatoly.solomin@cern.ch
>
>
  These should be done every time you want to start coding (NOTE the choice of User_release_area is completely up to you, actually, but should be in your AFS area):
Changed:
<
<
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser
  • mkdir /afs/cern.ch/user/a/asolomin/gituser
>
>
  • mkdir /afs/cern.ch/user/a/asolomin/gituser (where asolomin = your username)
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser (where asolomin = your username)
 
  • cd $User_release_area
Deleted:
<
<
  • git version (to check your git version, sadly there are version-dependent commands below)
  Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here I assume the newest is v6r0, you should check in gitlab):
  • lb-dev --nightly=lhcb-head Panoptes v6r0
Line: 128 to 135
 If after your first commit, you want to edit or expand your commit description before a push use:
  • git commit --amend
Changed:
<
<
You will now give your branch a name. At this stage you may choose anything (but pick something unique that helps you keep track of things). Say you pick asolomin-20170123-RICHMirrorAdjust.
This will push your changes to the asolomin-20170123-RICHMirrorAdjust branch:
>
>
You will now give your branch a name. You could choose anything (but pick something unique!). However let's try to stick to the following prescription: username-YYYYMMDD-title (e.g. asolomin-20170123-RICHMirrorAdjust).
So for example, to push your changes to a new branch called asolomin-20170123-RICHMirrorAdjust:
 
  • git lb-push Panoptes asolomin-20170123-RICHMirrorAdjust

Then, submit a merge request for asolomin-20170123-RICHMirrorAdjust from the Panoptes gitlab page You can also use the hyperlink provided when you commit, though if it's a sensitive change you may want to look at your commit in gitlab first. NOTE: Your pushed branch (that contains your commit) MUST COMPILE locally to get accepted (though it may not compile against the nightlies).

Changed:
<
<
Important: if you noticed a mistake and immediately want to commit a corrected version, you should
>
>
Important: if you noticed a mistake, fix it, and immediately want to commit a corrected version to a new branch, you should
 
  • git lb-push Panoptes asolomin-20170123-RICHMirrorAdjust2 (i.e. any different name)
Then, submit a merge request for asolomin-20170123-RICHMirrorAdjust2, instead of asolomin-20170123-RICHMirrorAdjust
Added:
>
>
Now say you already made a merge request, then you could commit a fix and push it to the same branch for which you made the merge request. Before you start work on the fix however, make sure to change the first word of the title of your merge request to "WIP: " (work-in-progress). This instructs the Project manager to wait until you remove "WIP: " to merge the branch.
 If you no longer want the merge request to be accepted (e.g. it was done by mistake) then close the merge request. If the associated branch is no longer in use you may delete it from the Panoptes gitlab page.

To run

Revision 122017-03-21 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 85 to 85
 
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser
  • mkdir /afs/cern.ch/user/a/asolomin/gituser
  • cd $User_release_area
Added:
>
>
  • git version (to check your git version, sadly there are version-dependent commands below)
  Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here I assume the newest is v6r0, you should check in gitlab):
  • lb-dev --nightly=lhcb-head Panoptes v6r0
Line: 100 to 101
 Now you can edit the code. You would obviously not necessarily edit the same files, and of course you should use your editor of choice (e.g. nano, emacs, vim, etc...) or edit as in this example:
  • edit Rich/RichMirrAlign/src/RichMirrAlign* &
  • edit Rich/RichMirrAlign/doc/release_notes.html Rich/RichMirrAlign/CMakeLists.txt Rich/RichMirrAlign/cmt/requirements & (always update the release notes!, only change the other two if the package "version" needs to be updated)
Changed:
<
<
  • make -j 8 (check that your code compiles!)
>
>
Check that your code compiles:
  • make -j 8 (use 8 cores)

Get a list of files that changed:

  • git status or
  • git diff --name-only

If need be, or to be safe, check all of the changes (except whitespace changes)

  • git diff --ignore-blank-lines --ignore-space-change (only works in git version 1.8.4 or higher)
  • git diff --ignore-space-change (always works)
  For everything that changed and any new files use git add (e.g. for the previous example):
  • git add Rich/RichMirrAlign/src/RichMirrAlign.cpp
Line: 120 to 131
 You will now give your branch a name. At this stage you may choose anything (but pick something unique that helps you keep track of things). Say you pick asolomin-20170123-RICHMirrorAdjust.
This will push your changes to the asolomin-20170123-RICHMirrorAdjust branch:
  • git lb-push Panoptes asolomin-20170123-RICHMirrorAdjust
Added:
>
>
 Then, submit a merge request for asolomin-20170123-RICHMirrorAdjust from the Panoptes gitlab page
Changed:
<
<
NOTE: Your pushed branch (that contains your commit) MUST COMPILE to get accepted.
>
>
You can also use the hyperlink provided when you commit, though if it's a sensitive change you may want to look at your commit in gitlab first. NOTE: Your pushed branch (that contains your commit) MUST COMPILE locally to get accepted (though it may not compile against the nightlies).
  Important: if you noticed a mistake and immediately want to commit a corrected version, you should
  • git lb-push Panoptes asolomin-20170123-RICHMirrorAdjust2 (i.e. any different name)

Revision 112017-03-21 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 10 to 10
 

git

Changed:
<
<
We are no longer using SVN. It is time to git! Please read everything and please follow the workflow Anatoly describes. If you have any questions email Anatoly and cc the development mailing list. If Anatoly cannot answer them, email Jordi, and if all else fails then Marco Clemencic / lhcb-core-soft@cernNOSPAMPLEASE.ch
>
>
We are no longer using SVN. It is time to git! Please read everything and please follow the workflow Anatoly describes. If you have any questions email the alignment-development mailing list. If we cannot answer your question, email Jordi, and if all else fails then Marco Clemencic / lhcb-core-soft@cernNOSPAMPLEASE.ch
 

Standard documentation and other git stuff

Deleted:
<
<
Don't know if any of these are useful yet. Please read the sections below and then fill in information gaps with one of the references.
 
Changed:
<
<
  • References:
>
>
  • Other References (don't know how useful these are):
 
Line: 28 to 26
 

Transition of Panoptes from svn to git.

Changed:
<
<
    The Panoptes project has been moved to gitlab. https://gitlab.cern.ch/lhcb/Panoptes
>
>
The Panoptes project has been moved to gitlab. https://gitlab.cern.ch/lhcb/Panoptes
  The lhcb-rich-software egroup is linked to the gitlab group such that all members have "developer" access.

There is a major change in the workflow, since we cannot use tagged packages in git anymore.

Line: 52 to 49
 
  • Submit a merge request, so the new features are included in the master branch.

Please get used to the git workflow ASAP.

Deleted:
<
<
 

The JIRA rich project.

Changed:
<
<
Jordi has created a JIRA project, which you can find here. Jordi is not an expert in JIRA, so Jordi cannot guarantee that the configuration that he has done is ideal. Any suggestions are welcome.
>
>
Jordi has created a JIRA (project and) component group for (the RICH group and) the mirror alignment, which you can find here. Jordi is not an expert in JIRA, so Jordi cannot guarantee that the configuration that he has done is ideal. Any suggestions are welcome.
  The JIRA group does NOT impose any change in the workflow. We could keep managing issues by email, or external service, or keep track of your tasks by tweeting them... However, people who have used JIRA have a good experience in their opinion, so we have decided to use it.
Line: 68 to 64
  Members of lhcb-rich-software have "developers" access to the rich JIRA project. (Let Jordi know if you prefer that Jordi changes it to lhcb-rich.)
Changed:
<
<

Anatoly's example git workflow for Panoptes

>
>

Anatoly's example git workflow for Panoptes [or Rec]

 
Changed:
<
<
Anatoly was kind enough to provide us with his git workflow so far. He is still learning as are we, but this will certainly help you get started.
>
>
Anatoly was kind enough to provide us with his git workflow so far. He is still learning as are we, but this will certainly help you get started. Paras is keeping this updated as new information arrives.
 

To build

Changed:
<
<
Change user name and email below as appropriate
>
>
Change usernames and email addresses below as appropriate
 
Changed:
<
<
new shell ( e.g. ssh   -Y   asolomin@lxplusNOSPAMPLEASE.cern.ch )
>
>
[ Change projects throughout below, if instead of Panoptes you are working with e.g. Rec ]

Create a new shell ( e.g. ssh   -Y   asolomin@lxplusNOSPAMPLEASE.cern.ch )

  Only once and forever do these three things:
Changed:
<
<
=setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser = (the choice of User_release_area is completely up to you, actually)
cd  $User_release_area
>
>
These should be done every time you want to start coding (NOTE the choice of User_release_area is completely up to you, actually, but should be in your AFS area):
  • setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser
  • mkdir /afs/cern.ch/user/a/asolomin/gituser
  • cd $User_release_area
 
Changed:
<
<
This you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here I assume the newest is v6r0, you should check):
>
>
Then this you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here I assume the newest is v6r0, you should check in gitlab):
 
  • lb-dev   --nightly=lhcb-head   Panoptes   v6r0
Changed:
<
<
cd   $User_release_area/PanoptesDev_v6r0
git   lb-use Panoptes

not sure yet if you need to keep redoing this (we currently do, every time after our branches have been accepted into the master); procedure replicating "svn update" is still being established

  • git   lb-checkout   Panoptes/master   Rich/RichMirrAlign

Example (if you're editing code you would obviously not necessarily edit the same files, and of course use your editor of choice)

  • edit Rich/RichMirrAlign/src/RichMirrAlign* &
  • edit Rich/RichMirrAlign/doc/release_notes.html Rich/RichMirrAlign/CMakeLists.txt Rich/RichMirrAlign/cmt/requirements &
  • make -j 6 (check that your code compiles!)

For everything that changed and any new files use git add (e.g. for the previous example):

  • git   add   Rich/RichMirrAlign/src/RichMirrAlign.cpp
  • git   add   Rich/RichMirrAlign/src/RichMirrAlignFcn.cpp
>
>
These you do as long as you are working with the project and version in the previous line (NOTE the "Dev"):
  • cd $User_release_area/PanoptesDev_v6r0
  • git lb-use Panoptes

This you do only to checkout a package or update everything you have from master (e.g, do this overtime before you start work on a package and do this every time after our branches have been accepted into the master)

<!--a procedure replicating "svn update" is still being established-->
  • git lb-checkout Panoptes/master Rich/RichMirrAlign (or perhaps a different package)

Now you can edit the code. You would obviously not necessarily edit the same files, and of course you should use your editor of choice (e.g. nano, emacs, vim, etc...) or edit as in this example:

  • edit Rich/RichMirrAlign/src/RichMirrAlign* &
  • edit Rich/RichMirrAlign/doc/release_notes.html Rich/RichMirrAlign/CMakeLists.txt Rich/RichMirrAlign/cmt/requirements & (always update the release notes!, only change the other two if the package "version" needs to be updated)
  • make -j 8 (check that your code compiles!)

For everything that changed and any new files use git add (e.g. for the previous example):

  • git add Rich/RichMirrAlign/src/RichMirrAlign.cpp
  • git add Rich/RichMirrAlign/src/RichMirrAlignFcn.cpp
 
  • git   add   Rich/RichMirrAlign/src/RichMirrAlignFcn.h
Changed:
<
<
  • git   add   Rich/RichMirrAlign/doc/release_notes.html (Please always update the release notes!!!)
  • git   add   Rich/RichMirrAlign/CMakeLists.txt
  • git   add   Rich/RichMirrAlign/cmt/requirements
>
>
  • git add Rich/RichMirrAlign/doc/release_notes.html (Please always update the release notes!!!)
  • git add Rich/RichMirrAlign/CMakeLists.txt
  • git add Rich/RichMirrAlign/cmt/requirements
  Then make sure to update the commit message:
  • git commit -m 'RICH-9999 regularizationMode introduced.'
Line: 130 to 132
 

To run

new shell ( e.g. ssh   -Y   asolomin@lxplusNOSPAMPLEASE.cern.ch )

Changed:
<
<
setenv   User_release_area /afs/cern.ch/user/a/asolomin/gituser

cd   $User_release_area/PanoptesDev_v6r0
make   install

cd   /afs/cern.ch/user/a/asolomin/public/rich_align_test (or wherever your test is)

lb-run   PanoptesDev v6r0   RichMirrAlign.exe  Rich1MirrAlign_Mp6Wi8.0Fm5Mm0Sm0_online_Collision15_i0.conf

Example git workflow for Hlt

To build

setenv   User_release_area /afs/cern.ch/user/p/pnaik/gituser (the choice of User_release_area is completely up to you, actually)
cd  $User_release_area

This you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release):

  • lb-dev   Moore   v{latest} (you figure out what the latest version is, )

cd   $User_release_area/MooreDev_v25r3

git lb-use Hlt

not sure yet if you need to keep redoing this; procedure replicating "svn update" is still being established

  • git lb-checkout Hlt/2016-patches Hlt/Hlt1Settings

Example (if you're editing code you would obviously not necessarily edit the same files, and of course use your editor of choice)

  • nano Hlt/HltSettings/python/HltSettings/Physics_pp_Draft2016.py
  • nano Hlt/HltSettings/doc/release.notes
  • make -j 6 (check that your code compiles!)

For everything that changed and any new files use git add (e.g. for the previous example):

  • git   add   Hlt/HltSettings/python/HltSettings/Physics_pp_Draft2016.py
  • git add Hlt/HltSettings/doc/release.notes

Then make sure to update the commit message:

>
>
  • setenv   User_release_area /afs/cern.ch/user/a/asolomin/gituser
  • cd   $User_release_area/PanoptesDev_v6r0
  • make   install
  • cd   /afs/cern.ch/user/a/asolomin/public/rich_align_test (or wherever your test is)
  • lb-run   PanoptesDev v6r0   RichMirrAlign.exe  Rich1MirrAlign_Mp6Wi8.0Fm5Mm0Sm0_online_Collision15_i0.conf

Slight differences to the above for the Hlt project

Substitute the following above, where appropriate

  • lb-dev Moore v25r3 (or latest version, check in gitlab)
  • cd $User_release_area/MooreDev_v25r3
  • git lb-use Hlt
  • git lb-checkout Hlt/2016-patches Hlt/HltSettings (perhaps you will be asked by HLT Operations to use a different branch, or need to edit a different package)
  • nano Hlt/HltSettings/python/HltSettings/Physics_pp_Draft2016.py (example)
  • nano Hlt/HltSettings/doc/release.notes
  • make -j 8 (check that your code compiles!)
  • git add Hlt/HltSettings/python/HltSettings/Physics_pp_Draft2016.py
  • git add Hlt/HltSettings/doc/release.notes
 
  • git commit -m 'RICH-9999 Rich Mirror Line prescales reduced by a factor of 3'
Deleted:
<
<
RICH-9999 is the associated JIRA task, which automatically links the commit (or later, merge request) to the JIRA task

If after your first commit, you want to edit or expand your commit description before a push use:

 
  • git commit --amend

You will now give your branch a name. At this stage you may choose anything (but pick something like {your name}-{something describing your change} that helps you keep track of things). Say you pick pnaik-20160630-RICHMirrorAdjust.

Line: 188 to 165
  If you no longer want the merge request to be accepted (e.g. it was done by mistake) then close the merge request. If the associated branch is no longer in use you may delete it from the Hlt gitlab page.
Deleted:
<
<

Example git workflow for Rec

To build

setenv   User_release_area /afs/cern.ch/user/p/pnaik/gituser (the choice of User_release_area is completely up to you, actually)
cd  $User_release_area

This you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release):

  • lb-dev   Rec   v{latest} (you figure out what the latest version is, )

cd   $User_release_area/Rec_{latest}

git lb-use Rec

not sure yet if you need to keep redoing this; procedure replicating "svn update" is still being established

  • git lb-checkout Rec/master Rich/RichRecQC

Example (if you're editing code you would obviously not necessarily edit the same files, and of course use your editor of choice)

  • nano Rich/RichRecQC/whatever.py
  • nano Rich/RichRecQC/doc/release.notes
  • make -j 6 (check that your code compiles!)

For everything that changed and any new files use git add (e.g. for the previous example):

  • git add Rich/RichRecQC/whatever.py
  • git add Rich/RichRecQC/doc/release.notes

Then make sure to update the commit message:

  • git commit -m 'RICH-9999 Added new selection called "TightMirror" for the mirror alignment'
RICH-9999 is the associated JIRA task, which automatically links the commit (or later, merge request) to the JIRA task

If after your first commit, you want to edit or expand your commit description before a push use:

  • git commit --amend

You will now give your branch a name. At this stage you may choose anything (but pick something like {your name}-{something describing your change} that helps you keep track of things). Say you pick pnaik-20160630-RichRecQCv2rXX.
This will push your changes to the pnaik-20160630-RichRecQCv2rXX branch:

  • git lb-push Rec pnaik-20160630-RichRecQCv2rXX
Then, submit a merge request for pnaik-20160630-RichRecQCv2rXX from the Rec gitlab page. Create a merge request with the master branch. NOTE: Your pushed branch (that contains your commit) MUST COMPILE to get accepted.

Important: if you noticed a mistake and immediately want to commit a corrected version, you should

  • git lb-push Rec pnaik-20160630-RichRecQCv2rXX-2 (i.e. any different name)
Then, submit a merge request for pnaik-20160630-RichRecQCv2rXX-2, instead of pnaik-20160630-RichRecQCv2rXX

If you no longer want the merge request to be accepted (e.g. it was done by mistake) then close the merge request. If the associated branch is no longer in use you may delete it from the Rec gitlab page.

 

Further Instructions from Jordi, based on Paras's initial questions

1. You should not associate the master branch with the SVN trunk. In SVN, tagged packages were expected to work, and committers could add unstable stuff to the trunk. With the new workflow that we believe is reasonable for Panoptes, the master branch is expected to always be stable, as opposed to the SVN trunk. Unstable contributions to git should take place in branches. So, to summarize, in svn, new contributions go from an unstable trunk to stable branches and from there to release. In git, new contributions go from unstable branches to stable master and from there to release.

Line: 300 to 230
 
  • Q: How during running one could use another project’s package ( before this was possible by getpacking whatever package you want to use, then using e.g. SetupProject Panoptes vXrY --use Rich/RichAlignment ). This is useful, for example, if we work on Rich/RichAlignment but a new version of Rec has not yet been released. Something to figure out later though as hopefully this is a rare case.
    • A: No clue. But of course, our use case is a "multiproject" one (Panoptes + Brunel), and in some situations (e.g. in the Ganga case) we need a recipe for that.
Added:
>
>

Troubleshooting

Q: All of my commits have now been merged. I git diff, and the only file that has changed is called .git-lb-checkout ... what should I do?

A: Definitely do not leave it. If that is the only file that is a problem and all of your commits have been merged:

git checkout -- .git-lb-checkout

Actually this may be worth doing in any instance, but we have no idea yet what .git-lb-checkout actually does

 

Legacy repository information

In the cloned gitlab repository, https://gitlab.cern.ch/LHCb-SVN-mirrors/Panoptes,

Revision 102017-03-20 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 28 to 28
 

Transition of Panoptes from svn to git.

Changed:
<
<
The Panoptes project has been moved to gitlab. https://gitlab.cern.ch/LHCb-SVN-mirrors/Panoptes

In the cloned gitlab repository, all contributions to svn up to release v5r5 have been getpacked and committed, but no other history is kept about them. After release v5r5, all the history has been kept.

For reference, Jordi has kept a complete copy of the svn repository at https://gitlab.cern.ch/lhcb-rich/panoptes In this longer version, all the svn history is kept and, additionally, all tagged releases have been getpacked and attached at their corresponding point in history. This is not intended to be used in practice, just for reference. Jordi will update this repository just once in a while but not very frequently.

Jordi has created the gitlab group lhcb-rich, intended for rich related repositories. https://gitlab.cern.ch/groups/lhcb-rich

>
>
    The Panoptes project has been moved to gitlab. https://gitlab.cern.ch/lhcb/Panoptes
  The lhcb-rich-software egroup is linked to the gitlab group such that all members have "developer" access.

There is a major change in the workflow, since we cannot use tagged packages in git anymore.

Line: 46 to 37
  The main consequences are the following:

  • CmakeLists and cmt/requirements still need to have a package version number, these should be updated when necessary.
Changed:
<
<
  • However, packages don't have version tags anymore. From now on, a Panoptes project tag affects all its packages. What used to be "version v2r26 of Rich/RichOnlineMonitors" is now the "Rich/RichOnlineMonitors that comes with Panoptes v5r6."
>
>
  • However, packages don't have version tags anymore. From now on, a Panoptes project tag affects all its packages. What used to be "version v2r26 of Rich/RichOnlineMonitors" is now the "Rich/RichOnlineMonitors that comes with Panoptes v5r6."
 
  • The tag collector becomes obsolete from the point of view of Panoptes.
Changed:
<
<
  • Since we still need to release Panoptes versions, all packages should at least compile when they are added to the master branch of the git repository. For this reason, it is important to have meaningful development branches for stuff that is not yet stable, which will be merged once the new features are fully implemented and tested.
>
>
  • Since we still need to release Panoptes versions, all packages should at least compile when they are added to the master branch of the git repository. For this reason, it is important to have meaningful development branches for stuff that is not yet stable, which will be merged once the new features are fully implemented and tested.
  The recommended workflow to add features to packages is the following:
Line: 57 to 51
 
  • Push the branch to an upstream repository (e.g. a fork of the project in gitlab).
  • Submit a merge request, so the new features are included in the master branch.
Changed:
<
<
Please get used to the git workflow asap.
>
>
Please get used to the git workflow ASAP.

The JIRA rich project.

 
Changed:
<
<

Anatoly's example git workflow for Panoptes (modified by Paras)

>
>
Jordi has created a JIRA project, which you can find here. Jordi is not an expert in JIRA, so Jordi cannot guarantee that the configuration that he has done is ideal. Any suggestions are welcome.

The JIRA group does NOT impose any change in the workflow. We could keep managing issues by email, or external service, or keep track of your tasks by tweeting them... However, people who have used JIRA have a good experience in their opinion, so we have decided to use it.

In JIRA there are these things called "*components*", which correspond to different unrelated topics (or mostly unrelated) within a project. Jordi has created four software components: monitoring (for Panoptes issues), alignment (for mirror alignment), piquet (for piquet software tools) and reconstruction (for reconstruction software). Please make sure any new JIRA issues for the alignment are created with component alignment.

(In principle, JIRA can be useful for hardware projects as well, so feel free to create as many components as you want for hardware and for other software topics.)

Rich JIRA issues have references like RICH-XXXX, where XXXX is a number. If you mention a JIRA issue in a git commit, JIRA will automatically create a comment in its corresponding issue page, so please do this (example below).

Members of lhcb-rich-software have "developers" access to the rich JIRA project. (Let Jordi know if you prefer that Jordi changes it to lhcb-rich.)

Anatoly's example git workflow for Panoptes

  Anatoly was kind enough to provide us with his git workflow so far. He is still learning as are we, but this will certainly help you get started.
Line: 100 to 109
 
  • git   add   Rich/RichMirrAlign/cmt/requirements

Then make sure to update the commit message:

Changed:
<
<
  • git commit -m 'regularizationMode introduced.'
>
>
  • git commit -m 'RICH-9999 regularizationMode introduced.'
RICH-9999 is the associated JIRA task, which automatically links the commit (or later, merge request) to the JIRA task
  If after your first commit, you want to edit or expand your commit description before a push use:
  • git commit --amend
Changed:
<
<
You will now give your branch a name. At this stage you may choose anything (but pick something unique that helps you keep track of things). Say you pick mirralign23Anatoly.
This will push your changes to the mirralign23Anatoly branch:
  • git lb-push Panoptes mirralign23Anatoly
Then, submit a merge request for mirralign23Anatoly from the Panoptes gitlab page
>
>
You will now give your branch a name. At this stage you may choose anything (but pick something unique that helps you keep track of things). Say you pick asolomin-20170123-RICHMirrorAdjust.
This will push your changes to the asolomin-20170123-RICHMirrorAdjust branch:
  • git lb-push Panoptes asolomin-20170123-RICHMirrorAdjust
Then, submit a merge request for asolomin-20170123-RICHMirrorAdjust from the Panoptes gitlab page
 NOTE: Your pushed branch (that contains your commit) MUST COMPILE to get accepted.

Important: if you noticed a mistake and immediately want to commit a corrected version, you should

Changed:
<
<
  • git lb-push Panoptes mirralign24Anatoly (i.e. any different name)
Then, submit a merge request for mirralign24Anatoly, instead of mirralign23Anatoly
>
>
  • git lb-push Panoptes asolomin-20170123-RICHMirrorAdjust2 (i.e. any different name)
Then, submit a merge request for asolomin-20170123-RICHMirrorAdjust2, instead of asolomin-20170123-RICHMirrorAdjust

If you no longer want the merge request to be accepted (e.g. it was done by mistake) then close the merge request. If the associated branch is no longer in use you may delete it from the Panoptes gitlab page.

 

To run

Line: 129 to 141
 lb-run   PanoptesDev v6r0   RichMirrAlign.exe  Rich1MirrAlign_Mp6Wi8.0Fm5Mm0Sm0_online_Collision15_i0.conf
Changed:
<
<

Paras's example git workflow for Hlt

>
>

Example git workflow for Hlt

 

To build

Line: 157 to 169
 
  • git add Hlt/HltSettings/doc/release.notes

Then make sure to update the commit message:

Changed:
<
<
  • git commit -m 'Rich Mirror Line prescales reduced by a factor of 3'
>
>
  • git commit -m 'RICH-9999 Rich Mirror Line prescales reduced by a factor of 3'
RICH-9999 is the associated JIRA task, which automatically links the commit (or later, merge request) to the JIRA task
  If after your first commit, you want to edit or expand your commit description before a push use:
  • git commit --amend
Changed:
<
<
You will now give your branch a name. At this stage you may choose anything (but pick something like {your name}-{something describing your change} that helps you keep track of things). Say you pick pnaik-20160630-RICHMirrorAdjust2.
This will push your changes to the pnaik-20160630-RICHMirrorAdjust2 branch:
  • git lb-push Hlt pnaik-20160630-RICHMirrorAdjust2
Then, submit a merge request for pnaik-20160630-RICHMirrorAdjust2 from the Hlt gitlab page. Create a merge request with the 2016-patches branch.
>
>
You will now give your branch a name. At this stage you may choose anything (but pick something like {your name}-{something describing your change} that helps you keep track of things). Say you pick pnaik-20160630-RICHMirrorAdjust.
This will push your changes to the pnaik-20160630-RICHMirrorAdjust branch:
  • git lb-push Hlt pnaik-20160630-RICHMirrorAdjust
Then, submit a merge request for pnaik-20160630-RICHMirrorAdjust from the Hlt gitlab page. Create a merge request with the 2016-patches branch.
 NOTE: Your pushed branch (that contains your commit) MUST COMPILE to get accepted.
Changed:
<
<

Paras's example git workflow for Rec

>
>
Important: if you noticed a mistake and immediately want to commit a corrected version, you should
  • git lb-push Hlt pnaik-20160630-RICHMirrorAdjust2 (i.e. any different name)
Then, submit a merge request for pnaik-20160630-RICHMirrorAdjust2, instead of pnaik-20160630-RICHMirrorAdjust

If you no longer want the merge request to be accepted (e.g. it was done by mistake) then close the merge request. If the associated branch is no longer in use you may delete it from the Hlt gitlab page.

Example git workflow for Rec

 

To build

Line: 196 to 216
 
  • git add Rich/RichRecQC/doc/release.notes

Then make sure to update the commit message:

Changed:
<
<
  • git commit -m 'Added new selection called "TightMirror" for the mirror alignment'
>
>
  • git commit -m 'RICH-9999 Added new selection called "TightMirror" for the mirror alignment'
RICH-9999 is the associated JIRA task, which automatically links the commit (or later, merge request) to the JIRA task
  If after your first commit, you want to edit or expand your commit description before a push use:
  • git commit --amend
Line: 204 to 225
 You will now give your branch a name. At this stage you may choose anything (but pick something like {your name}-{something describing your change} that helps you keep track of things). Say you pick pnaik-20160630-RichRecQCv2rXX.
This will push your changes to the pnaik-20160630-RichRecQCv2rXX branch:
  • git lb-push Rec pnaik-20160630-RichRecQCv2rXX
Changed:
<
<
Then, submit a merge request for pnaik-20160630-RichRecQCv2rXX from the Hlt gitlab page. Create a merge request with the master branch.
>
>
Then, submit a merge request for pnaik-20160630-RichRecQCv2rXX from the Rec gitlab page. Create a merge request with the master branch.
 NOTE: Your pushed branch (that contains your commit) MUST COMPILE to get accepted.
Added:
>
>
Important: if you noticed a mistake and immediately want to commit a corrected version, you should
  • git lb-push Rec pnaik-20160630-RichRecQCv2rXX-2 (i.e. any different name)
Then, submit a merge request for pnaik-20160630-RichRecQCv2rXX-2, instead of pnaik-20160630-RichRecQCv2rXX

If you no longer want the merge request to be accepted (e.g. it was done by mistake) then close the merge request. If the associated branch is no longer in use you may delete it from the Rec gitlab page.

 

Further Instructions from Jordi, based on Paras's initial questions

1. You should not associate the master branch with the SVN trunk. In SVN, tagged packages were expected to work, and committers could add unstable stuff to the trunk. With the new workflow that we believe is reasonable for Panoptes, the master branch is expected to always be stable, as opposed to the SVN trunk. Unstable contributions to git should take place in branches. So, to summarize, in svn, new contributions go from an unstable trunk to stable branches and from there to release. In git, new contributions go from unstable branches to stable master and from there to release.

Line: 245 to 273
 git fetch Panoptes
git lb-checkout Panoptes/that-branch Rich/RichMirrorAlignmentOnline
Changed:
<
<
You can also tag specific commits on the branch, so you can easily refer to them with others. Please don't use v-tags: the Panoptes project should be the only thing that is v-tagged. If v-tags help you understand better where you are, please prefix them with some short context, e.g. align-v1r0.
>
>
You can also tag specific commits on the branch, so you can easily refer to them with others. Please don't use v-tags: the Panoptes project should be the only thing that is v-tagged. If v-tags help you understand better where you are, please prefix them with some short context, e.g. align-v1r0. Basically the only place a package v-tag should show up is within the CMakeLists and cmt/requirements of each package
  3. I think the lb-use and lb-checkout commands also solve the case where you want to use packages as plugins from other projects. You can very consistently do
Line: 256 to 284
  4. About releasing a new version of Panoptes every time a package has a substantial improvement: I don't have a correct answer to this question. New Panoptes releases are created when there's interest in them. It may be for several reasons: to have new features released, to use a new release of Online, to compile with a new CMTCONFIG,... I never oppose creating a new release if it is requested.
Changed:
<
<
5. Who approves merge requests? When the svn repository becomes read-only, it will make sense to designate who should be allowed to accept them. [Note: This has happened; still don't know who this is]
>
>
5. Who approves merge requests? Currently Jordi Garra Tico and Chris Jones.
 
Added:
>
>
6. Git add must be done immediately before a git commit/lb-push; if any file changes git add must be done again.
 

List of questions to be asked and answered (by Jordi, etc...) at an appropriate time

Changed:
<
<
What is the reason to commit (but without a push) the fact that a package was checked out?

Jordi talked about us being able to create some sort of “local” tag, like “mirralign-RichMirrAlign-v19r2”. How do we do that, as the unique names given to every branch [such as "mirralign23"] exist only between your push and the moment when Jordi (with approval from Chris) "merges" it to the master. (Of course, Jordi does the merge only after you "asked" for it online.)

>
>
What is the reason to commit (but without a push) the fact that a package was checked out? [*NOTE* No one has ever complained that we didn't...]
  Also, is there a way, say after you commit “mirralign23”, to do something similar to “svn update” to get your local branch to be in sync with the master branch (say, if your changes are accepted, then someone else changes it), or would you have to check out the head version each time?

Questions on which Anatoly was not 100% sure:

Deleted:
<
<
  • Q: Git add must be done immediately before a git commit/lb-push and if any file changes git add must be done again?
    • A: not exactly: "adding" is just formation of the list of files to be gitted, so you can do that any time. However, I'm not 100% sure, so please keep this Q for the experts...
 
  • Q: You cannot "git lb-push" to a branch already existing in the remote repository, unless you checked out from that specific branch?
    • A: I think, it is without "unless", meaning that such a branch is always unique, but keep this Q too.
  • Q: How during running one could use another project’s package ( before this was possible by getpacking whatever package you want to use, then using e.g. SetupProject Panoptes vXrY --use Rich/RichAlignment ). This is useful, for example, if we work on Rich/RichAlignment but a new version of Rec has not yet been released. Something to figure out later though as hopefully this is a rare case.
    • A: No clue. But of course, our use case is a "multiproject" one (Panoptes + Brunel), and in some situations (e.g. in the Ganga case) we need a recipe for that.
Changed:
<
<

The JIRA rich project.

Jordi has created a jira project, which you can find here. Jordi is not an expert in jira, so Jordi cannot guarantee that the configuration that he has done is ideal. If you know more than Jordi about project management (which is very likely), please help create a nice framework to handle rich issues.

The jira group does NOT impose any change in the workflow. If you prefer to keep managing issues by email, or if you use an external service, or even if you keep track of your tasks by tweeting them, this is still fine. However, people who have used jira have a good experience in their opinion, so it may be worth giving it a try.

In jira there are these things called "components", which correspond to different unrelated topics (or mostly unrelated) within a project. I have created four software components: monitoring (for Panoptes issues), alignment (for mirror alignment), piquet (for piquet software tools) and reconstruction (for reconstruction software). In principle, jira can be useful for hardware projects as well, so feel free to create as many components as you want for hardware and for other software topics.

Rich jira issues have references like RICH-XXXX, where XXXX is a number. If you mention a jira issue in a git commit, jira will automatically create a comment in its corresponding issue page.

Members of lhcb-rich-software have "developers" access to the rich jira project. Let Jordi know if you prefer that Jordi changes it to lhcb-rich.

>
>

Legacy repository information

 
Added:
>
>
In the cloned gitlab repository, https://gitlab.cern.ch/LHCb-SVN-mirrors/Panoptes, all contributions to svn up to release v5r5 have been getpacked and committed, but no other history is kept about them. After release v5r5, all the history has been kept.
 
Added:
>
>
For reference, Jordi has kept a complete copy of the SVN repository at https://gitlab.cern.ch/lhcb-rich/panoptes In this longer version, all the svn history is kept and, additionally, all tagged releases have been getpacked and attached at their corresponding point in history. This is not intended to be used in practice, just for reference. Jordi will update this repository just once in a while but not very frequently.
 
Added:
>
>
Jordi has created the gitlab group lhcb-rich, intended for rich related repositories. https://gitlab.cern.ch/groups/lhcb-rich
 

The End

Revision 92017-03-17 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 203 to 203
  You will now give your branch a name. At this stage you may choose anything (but pick something like {your name}-{something describing your change} that helps you keep track of things). Say you pick pnaik-20160630-RichRecQCv2rXX.
This will push your changes to the pnaik-20160630-RichRecQCv2rXX branch:
Changed:
<
<
  • git lb-push Hlt pnaik-20160630-RichRecQCv2rXX
>
>
  • git lb-push Rec pnaik-20160630-RichRecQCv2rXX
 Then, submit a merge request for pnaik-20160630-RichRecQCv2rXX from the Hlt gitlab page. Create a merge request with the master branch. NOTE: Your pushed branch (that contains your commit) MUST COMPILE to get accepted.

Revision 82017-03-16 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 41 to 41
  Jordi has created the gitlab group lhcb-rich, intended for rich related repositories. https://gitlab.cern.ch/groups/lhcb-rich The lhcb-rich-software egroup is linked to the gitlab group such that all members have "developer" access.
Deleted:
<
<
<!--    The gitlab and svn repositories are synced every 30 minutes, in both directions: 
      if you commit something to svn, it will show up in the gitlab repository, and accepted merge requests in gitlab will appear in svn. -->
  There is a major change in the workflow, since we cannot use tagged packages in git anymore.

The main consequences are the following:

  • CmakeLists and cmt/requirements still need to have a package version number, these should be updated when necessary.
Changed:
<
<
  • However, packages don't have version tags anymore. From now on, a Panoptes project tag affects all its packages. What used to be "version v2r26 of Rich/RichOnlineMonitors" is now the "Rich/RichOnlineMonitors that comes with Panoptes v5r6."
    <!-- This is also true even if you tag a package in svn: no package version information will be propagated to the gitlab repository, anyway, so it will not be used when the project is released. -->
>
>
  • However, packages don't have version tags anymore. From now on, a Panoptes project tag affects all its packages. What used to be "version v2r26 of Rich/RichOnlineMonitors" is now the "Rich/RichOnlineMonitors that comes with Panoptes v5r6."
 
  • The tag collector becomes obsolete from the point of view of Panoptes.
  • Since we still need to release Panoptes versions, all packages should at least compile when they are added to the master branch of the git repository. For this reason, it is important to have meaningful development branches for stuff that is not yet stable, which will be merged once the new features are fully implemented and tested.
Line: 60 to 57
 
  • Push the branch to an upstream repository (e.g. a fork of the project in gitlab).
  • Submit a merge request, so the new features are included in the master branch.
Deleted:
<
<
<!-- You may have figured out a way to cheat with the repositories: if you commit your changes to svn, the branch that will be automatically created in gitlab will be merged anyway, without having to go through a merge request. This may sound like a nice workaround to not having to change the way you work, but it actually means breaking the workflow through svn and maybe introducing unwanted changes to the master branch, which is not for development as the svn trunk used to be. This is not intended behavior and, for this reason, after some short coexistence time, the plan is to make the svn repo read-only in the short term. Please report any inconveniences that this may cause you before this happens. We all have our scripts in place, we're not aware of the scripts that other people use and, if they may break, it's better to not find out by surprise. -->
  Please get used to the git workflow asap.
Deleted:
<
<
<!--
    You can use the git commands introduced by Marco some time ago. As an example:

lb-dev Panoptes v5r6 cd PanoptesDev_v5r6 git lb-use Panoptes git lb-checkout Panoptes/v5r6 Rich/Panoptes [... work on Rich/Panoptes, and similarly for any other Panoptes package ...] git lb-push Panoptes branch-name [... submit a merge request ...] -->

<!--

Paras's Email back to Jordi

Hi Jordi (and removing lhcb-rich for now),

Congratulations on getting this all into git! I will take your advice to start using it right away, but since I have not used git I cannot predict anything that I am about to do wrong smile At the moment I am a little lost about development branches and master branches. If the master branch is the trunk, development branches are the local copies, and merge requests are commits, then I get it already, but if I have that wrong or there are subtleties then I will learn.

Speaking only to our experience in the mirror alignment (but maybe others share this), we are used to checking out the package head locally using svn, checking for svn updates often to avoid falling behind on each other’s changes, committing often (only when code compiles of course), updating the release notes, and and tagging code as a new version when appropriate. So if these habits are still the right ones for packages in git (except maybe the tagging), we will probably be fine.

The ability to have our packages in project Panoptes, but be able to be used as plug-ins to other projects (e.g. AlignmentOnline), has worked very well for us. Keeping track of major improvements via package tags and the tag collector thus has been a nice way to keep track of major improvements; and we would know for certain that only a tagged package would go into any new version of the project created by the project manager.

Since we are moving from tagged packages and projects to tagged projects only, would it make sense to then release a new version of Panoptes every time a package of Panoptes has a substantial improvement? If this is getting carried away, then how else should we internally keep track of our package “tags”? For example we could use svn revision number, but I suppose git will have its own revision numbers too?

Cheers,

~ Paras

P.S. OK I am cheating on my promise to go learn here but I am too curious… do merge requests need to be approved, or do they just happen (like commits)? If they need to be approved, who approves them? P.P.S. Thanks for starting the JIRA, we probably should have used it long ago for the mirror alignment rather than keep track of everything on our TWiki. We will try to migrate our open issues to JIRA soon. -->

Further Instructions from Jordi, based on Paras's initial questions

1. You should not associate the master branch with the SVN trunk. In SVN, tagged packages were expected to work, and committers could add unstable stuff to the trunk. With the new workflow that we believe is reasonable for Panoptes, the master branch is expected to always be stable, as opposed to the SVN trunk. Unstable contributions to git should take place in branches. So, to summarize, in svn, new contributions go from an unstable trunk to stable branches and from there to release. In git, new contributions go from unstable branches to stable master and from there to release.

2. An example git workflow :

lb-dev Panoptes v5r6

This creates a local git repository in PanoptesDev_v5r6. This is called a "satellite project". Then,

cd PanoptesDev_v5r6
git lb-use Panoptes

This adds the gitlab remote to the local repository of the satellite project, and fetches that remote (makes a local copy of the remote repository, although you still don't see it in your working tree).

The output of this last command will give you information like this:

* [new branch] master -> Panoptes/master
* [new tag] v1r0 -> Panoptes/v1r0
[...]
* [new tag] v5r5 -> Panoptes/v5r5
* [new tag] v5r6 -> Panoptes/v5r6

You will see that there are several references that you can use. Some of them are branches (right now only master) and some of them are tags. You will refer to them as Panoptes/[reference], as in the last column of the previous output.

Lets say you want to develop on top of the Rich/RichMirrorAlignmentOnline package that you can find on the master branch. You would do

git lb-checkout Panoptes/master Rich/RichMirrorAlignmentOnline

This is the equivalent of checking out the package from the head. Then you would add new features to the mirror alignment software, and maybe you will commit unfinished work that you want to share with others. In this case, you would do

git lb-push Panoptes newfeature

Here, newfeature is the name of your development branch, and you will be able to see it in gitlab. It should be short, but descriptive. If someone else has lb-pushed a new feature branch to Panoptes, you can bring it to your local copy with

git fetch Panoptes
git lb-checkout Panoptes/that-branch Rich/RichMirrorAlignmentOnline

 
Changed:
<
<
You can also tag specific commits on the branch, so you can easily refer to them with others. Please don't use v-tags: the Panoptes project should be the only thing that is v-tagged. If v-tags help you understand better where you are, please prefix them with some short context, e.g. align-v1r0.

3. I think the lb-use and lb-checkout commands also solve the case where you want to use packages as plugins from other projects. You can very consistently do

git lb-use DaVinci
git lb-checkout DaVinci/v38r1 Phys/DaVinci

So this is how you would use the Phys/DaVinci package in Panoptes (regardless of the fact that it would not make any sense).

4. About releasing a new version of Panoptes every time a package has a substantial improvement: I don't have a correct answer to this question. New Panoptes releases are created when there's interest in them. It may be for several reasons: to have new features released, to use a new release of Online, to compile with a new CMTCONFIG,... I never oppose creating a new release if it is requested.

5. Who approves merge requests?

<!--Right now, in the period of coexistence, they are automatically approved by the librarian account, liblhcb. -->
When the svn repository becomes read-only, it will make sense to designate who should be allowed to accept them. [Note: still don't know who this is]

Anatoly's example git workflow for Panoptes

>
>

Anatoly's example git workflow for Panoptes (modified by Paras)

  Anatoly was kind enough to provide us with his git workflow so far. He is still learning as are we, but this will certainly help you get started.

To build

Added:
>
>
Change user name and email below as appropriate
 new shell ( e.g. ssh   -Y   asolomin@lxplusNOSPAMPLEASE.cern.ch )
Changed:
<
<
Only once and forever:
>
>
Only once and forever do these three things:
 
Changed:
<
<
setenv   User_release_area /afs/cern.ch/user/a/asolomin/public/rich_align (the choice of User_release_area is completely up to you, actually; Paras recommends e.g. /afs/cern.ch/user/p/pnaik/gituser)
>
>
=setenv User_release_area /afs/cern.ch/user/a/asolomin/gituser = (the choice of User_release_area is completely up to you, actually)
 cd  $User_release_area
Changed:
<
<
This you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release):
>
>
This you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release, here I assume the newest is v6r0, you should check):
 
  • lb-dev   --nightly=lhcb-head   Panoptes   v6r0

cd   $User_release_area/PanoptesDev_v6r0
git   lb-use Panoptes

Changed:
<
<
not sure yet if you need to keep redoing this; procedure replicating "svn update" is still being established
>
>
not sure yet if you need to keep redoing this (we currently do, every time after our branches have been accepted into the master); procedure replicating "svn update" is still being established
 
  • git   lb-checkout   Panoptes/master   Rich/RichMirrAlign
Deleted:
<
<
<!-- git   commit   -m 'imported Rich/RichMirrAlign from Panoptes' -->
  Example (if you're editing code you would obviously not necessarily edit the same files, and of course use your editor of choice)
  • edit Rich/RichMirrAlign/src/RichMirrAlign* &
Line: 206 to 119
  new shell ( e.g. ssh   -Y   asolomin@lxplusNOSPAMPLEASE.cern.ch )
Changed:
<
<
setenv   User_release_area /afs/cern.ch/user/a/asolomin/public/rich_align
>
>
setenv   User_release_area /afs/cern.ch/user/a/asolomin/gituser
  cd   $User_release_area/PanoptesDev_v6r0
make   install
Line: 215 to 128
  lb-run   PanoptesDev v6r0   RichMirrAlign.exe  Rich1MirrAlign_Mp6Wi8.0Fm5Mm0Sm0_online_Collision15_i0.conf
Deleted:
<
<

List of questions to be asked and answered (by Jordi, etc...) at an appropriate time

What is the reason to commit (but without a push) the fact that a package was checked out?

Jordi talked about us being able to create some sort of “local” tag, like “mirralign-RichMirrAlign-v19r2”. How do we do that, as the unique names given to every branch [such as "mirralign23"] exist only between your push and the moment when Jordi (with approval from Chris) "merges" it to the master. (Of course, Jordi does the merge only after you "asked" for it online.)

Also, is there a way, say after you commit “mirralign23”, to do something similar to “svn update” to get your local branch to be in sync with the master branch (say, if your changes are accepted, then someone else changes it), or would you have to check out the head version each time?

Questions on which Anatoly was not 100% sure:

  • Q: Git add must be done immediately before a git commit/lb-push and if any file changes git add must be done again?
    • A: not exactly: "adding" is just formation of the list of files to be gitted, so you can do that any time. However, I'm not 100% sure, so please keep this Q for the experts...
  • Q: You cannot "git lb-push" to a branch already existing in the remote repository, unless you checked out from that specific branch?
    • A: I think, it is without "unless", meaning that such a branch is always unique, but keep this Q too.
  • Q: How during running one could use another project’s package ( before this was possible by getpacking whatever package you want to use, then using e.g. SetupProject Panoptes vXrY --use Rich/RichAlignment ). This is useful, for example, if we work on Rich/RichAlignment but a new version of Rec has not yet been released. Something to figure out later though as hopefully this is a rare case.
    • A: No clue. But of course, our use case is a "multiproject" one (Panoptes + Brunel), and in some situations (e.g. in the Ganga case) we need a recipe for that.

The JIRA rich project.

Jordi has created a jira project, which you can find here. Jordi is not an expert in jira, so Jordi cannot guarantee that the configuration that he has done is ideal. If you know more than Jordi about project management (which is very likely), please help create a nice framework to handle rich issues.

The jira group does NOT impose any change in the workflow. If you prefer to keep managing issues by email, or if you use an external service, or even if you keep track of your tasks by tweeting them, this is still fine. However, people who have used jira have a good experience in their opinion, so it may be worth giving it a try.

In jira there are these things called "components", which correspond to different unrelated topics (or mostly unrelated) within a project. I have created four software components: monitoring (for Panoptes issues), alignment (for mirror alignment), piquet (for piquet software tools) and reconstruction (for reconstruction software). In principle, jira can be useful for hardware projects as well, so feel free to create as many components as you want for hardware and for other software topics.

Rich jira issues have references like RICH-XXXX, where XXXX is a number. If you mention a jira issue in a git commit, jira will automatically create a comment in its corresponding issue page.

Members of lhcb-rich-software have "developers" access to the rich jira project. Let Jordi know if you prefer that Jordi changes it to lhcb-rich.

 

Paras's example git workflow for Hlt

Line: 282 to 168
 Then, submit a merge request for pnaik-20160630-RICHMirrorAdjust2 from the Hlt gitlab page. Create a merge request with the 2016-patches branch. NOTE: Your pushed branch (that contains your commit) MUST COMPILE to get accepted.
Added:
>
>

Paras's example git workflow for Rec

To build

setenv   User_release_area /afs/cern.ch/user/p/pnaik/gituser (the choice of User_release_area is completely up to you, actually)
cd  $User_release_area

This you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release):

  • lb-dev   Rec   v{latest} (you figure out what the latest version is, )

cd   $User_release_area/Rec_{latest}

git lb-use Rec

not sure yet if you need to keep redoing this; procedure replicating "svn update" is still being established

  • git lb-checkout Rec/master Rich/RichRecQC

Example (if you're editing code you would obviously not necessarily edit the same files, and of course use your editor of choice)

  • nano Rich/RichRecQC/whatever.py
  • nano Rich/RichRecQC/doc/release.notes
  • make -j 6 (check that your code compiles!)

For everything that changed and any new files use git add (e.g. for the previous example):

  • git add Rich/RichRecQC/whatever.py
  • git add Rich/RichRecQC/doc/release.notes

Then make sure to update the commit message:

  • git commit -m 'Added new selection called "TightMirror" for the mirror alignment'

If after your first commit, you want to edit or expand your commit description before a push use:

  • git commit --amend

You will now give your branch a name. At this stage you may choose anything (but pick something like {your name}-{something describing your change} that helps you keep track of things). Say you pick pnaik-20160630-RichRecQCv2rXX.
This will push your changes to the pnaik-20160630-RichRecQCv2rXX branch:

  • git lb-push Hlt pnaik-20160630-RichRecQCv2rXX
Then, submit a merge request for pnaik-20160630-RichRecQCv2rXX from the Hlt gitlab page. Create a merge request with the master branch. NOTE: Your pushed branch (that contains your commit) MUST COMPILE to get accepted.

Further Instructions from Jordi, based on Paras's initial questions

1. You should not associate the master branch with the SVN trunk. In SVN, tagged packages were expected to work, and committers could add unstable stuff to the trunk. With the new workflow that we believe is reasonable for Panoptes, the master branch is expected to always be stable, as opposed to the SVN trunk. Unstable contributions to git should take place in branches. So, to summarize, in svn, new contributions go from an unstable trunk to stable branches and from there to release. In git, new contributions go from unstable branches to stable master and from there to release.

2. An example git workflow :

lb-dev Panoptes v5r6

This creates a local git repository in PanoptesDev_v5r6. This is called a "satellite project". Then,

cd PanoptesDev_v5r6
git lb-use Panoptes

This adds the gitlab remote to the local repository of the satellite project, and fetches that remote (makes a local copy of the remote repository, although you still don't see it in your working tree).

The output of this last command will give you information like this:

* [new branch] master -> Panoptes/master
* [new tag] v1r0 -> Panoptes/v1r0
[...]
* [new tag] v5r5 -> Panoptes/v5r5
* [new tag] v5r6 -> Panoptes/v5r6

You will see that there are several references that you can use. Some of them are branches (right now only master) and some of them are tags. You will refer to them as Panoptes/[reference], as in the last column of the previous output.

Lets say you want to develop on top of the Rich/RichMirrorAlignmentOnline package that you can find on the master branch. You would do

git lb-checkout Panoptes/master Rich/RichMirrorAlignmentOnline

This is the equivalent of checking out the package from the head. Then you would add new features to the mirror alignment software, and maybe you will commit unfinished work that you want to share with others. In this case, you would do

git lb-push Panoptes newfeature

Here, newfeature is the name of your development branch, and you will be able to see it in gitlab. It should be short, but descriptive. If someone else has lb-pushed a new feature branch to Panoptes, you can bring it to your local copy with

git fetch Panoptes
git lb-checkout Panoptes/that-branch Rich/RichMirrorAlignmentOnline

You can also tag specific commits on the branch, so you can easily refer to them with others. Please don't use v-tags: the Panoptes project should be the only thing that is v-tagged. If v-tags help you understand better where you are, please prefix them with some short context, e.g. align-v1r0.

3. I think the lb-use and lb-checkout commands also solve the case where you want to use packages as plugins from other projects. You can very consistently do

git lb-use DaVinci
git lb-checkout DaVinci/v38r1 Phys/DaVinci

So this is how you would use the Phys/DaVinci package in Panoptes (regardless of the fact that it would not make any sense).

4. About releasing a new version of Panoptes every time a package has a substantial improvement: I don't have a correct answer to this question. New Panoptes releases are created when there's interest in them. It may be for several reasons: to have new features released, to use a new release of Online, to compile with a new CMTCONFIG,... I never oppose creating a new release if it is requested.

5. Who approves merge requests? When the svn repository becomes read-only, it will make sense to designate who should be allowed to accept them. [Note: This has happened; still don't know who this is]

List of questions to be asked and answered (by Jordi, etc...) at an appropriate time

What is the reason to commit (but without a push) the fact that a package was checked out?

Jordi talked about us being able to create some sort of “local” tag, like “mirralign-RichMirrAlign-v19r2”. How do we do that, as the unique names given to every branch [such as "mirralign23"] exist only between your push and the moment when Jordi (with approval from Chris) "merges" it to the master. (Of course, Jordi does the merge only after you "asked" for it online.)

Also, is there a way, say after you commit “mirralign23”, to do something similar to “svn update” to get your local branch to be in sync with the master branch (say, if your changes are accepted, then someone else changes it), or would you have to check out the head version each time?

Questions on which Anatoly was not 100% sure:

  • Q: Git add must be done immediately before a git commit/lb-push and if any file changes git add must be done again?
    • A: not exactly: "adding" is just formation of the list of files to be gitted, so you can do that any time. However, I'm not 100% sure, so please keep this Q for the experts...
  • Q: You cannot "git lb-push" to a branch already existing in the remote repository, unless you checked out from that specific branch?
    • A: I think, it is without "unless", meaning that such a branch is always unique, but keep this Q too.
  • Q: How during running one could use another project’s package ( before this was possible by getpacking whatever package you want to use, then using e.g. SetupProject Panoptes vXrY --use Rich/RichAlignment ). This is useful, for example, if we work on Rich/RichAlignment but a new version of Rec has not yet been released. Something to figure out later though as hopefully this is a rare case.
    • A: No clue. But of course, our use case is a "multiproject" one (Panoptes + Brunel), and in some situations (e.g. in the Ganga case) we need a recipe for that.

The JIRA rich project.

Jordi has created a jira project, which you can find here. Jordi is not an expert in jira, so Jordi cannot guarantee that the configuration that he has done is ideal. If you know more than Jordi about project management (which is very likely), please help create a nice framework to handle rich issues.

The jira group does NOT impose any change in the workflow. If you prefer to keep managing issues by email, or if you use an external service, or even if you keep track of your tasks by tweeting them, this is still fine. However, people who have used jira have a good experience in their opinion, so it may be worth giving it a try.

In jira there are these things called "components", which correspond to different unrelated topics (or mostly unrelated) within a project. I have created four software components: monitoring (for Panoptes issues), alignment (for mirror alignment), piquet (for piquet software tools) and reconstruction (for reconstruction software). In principle, jira can be useful for hardware projects as well, so feel free to create as many components as you want for hardware and for other software topics.

Rich jira issues have references like RICH-XXXX, where XXXX is a number. If you mention a jira issue in a git commit, jira will automatically create a comment in its corresponding issue page.

Members of lhcb-rich-software have "developers" access to the rich jira project. Let Jordi know if you prefer that Jordi changes it to lhcb-rich.

 

The End

-- ParasNaik - 2016-04-12 \ No newline at end of file

Revision 72017-03-16 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 160 to 160
 
  • git config --global user.name "Anatoly Solomin"
  • git config --global user.email anatoly.solomin@cern.ch
Changed:
<
<
setenv   User_release_area /afs/cern.ch/user/a/asolomin/public/rich_align (the choice of User_release_area is completely up to you, actually)
>
>
setenv   User_release_area /afs/cern.ch/user/a/asolomin/public/rich_align (the choice of User_release_area is completely up to you, actually; Paras recommends e.g. /afs/cern.ch/user/p/pnaik/gituser)
 cd  $User_release_area

This you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release):

Revision 62016-06-30 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 147 to 147
  5. Who approves merge requests?
<!--Right now, in the period of coexistence, they are automatically approved by the librarian account, liblhcb. -->
When the svn repository becomes read-only, it will make sense to designate who should be allowed to accept them. [Note: still don't know who this is]
Changed:
<
<

Anatoly's example git workflow

>
>

Anatoly's example git workflow for Panoptes

  Anatoly was kind enough to provide us with his git workflow so far. He is still learning as are we, but this will certainly help you get started.
Line: 243 to 243
  Members of lhcb-rich-software have "developers" access to the rich jira project. Let Jordi know if you prefer that Jordi changes it to lhcb-rich.
Added:
>
>

Paras's example git workflow for Hlt

To build

setenv   User_release_area /afs/cern.ch/user/p/pnaik/gituser (the choice of User_release_area is completely up to you, actually)
cd  $User_release_area

This you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release):

  • lb-dev   Moore   v{latest} (you figure out what the latest version is, )

cd   $User_release_area/MooreDev_v25r3

git lb-use Hlt

not sure yet if you need to keep redoing this; procedure replicating "svn update" is still being established

  • git lb-checkout Hlt/2016-patches Hlt/Hlt1Settings

Example (if you're editing code you would obviously not necessarily edit the same files, and of course use your editor of choice)

  • nano Hlt/HltSettings/python/HltSettings/Physics_pp_Draft2016.py
  • nano Hlt/HltSettings/doc/release.notes
  • make -j 6 (check that your code compiles!)

For everything that changed and any new files use git add (e.g. for the previous example):

  • git   add   Hlt/HltSettings/python/HltSettings/Physics_pp_Draft2016.py
  • git add Hlt/HltSettings/doc/release.notes

Then make sure to update the commit message:

  • git commit -m 'Rich Mirror Line prescales reduced by a factor of 3'

If after your first commit, you want to edit or expand your commit description before a push use:

  • git commit --amend

You will now give your branch a name. At this stage you may choose anything (but pick something like {your name}-{something describing your change} that helps you keep track of things). Say you pick pnaik-20160630-RICHMirrorAdjust2.
This will push your changes to the pnaik-20160630-RICHMirrorAdjust2 branch:

  • git lb-push Hlt pnaik-20160630-RICHMirrorAdjust2
Then, submit a merge request for pnaik-20160630-RICHMirrorAdjust2 from the Hlt gitlab page. Create a merge request with the 2016-patches branch. NOTE: Your pushed branch (that contains your commit) MUST COMPILE to get accepted.
 

The End

-- ParasNaik - 2016-04-12

Revision 52016-06-21 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 10 to 10
 

git

Changed:
<
<
We are no longer using SVN. It is time to git! Please read everything and please follow the workflow Anatoly describes. If you have any questions email Anatoly and cc the development mailing list. If Anatoly cannot answer them, email Jordi, and if all else fails then Marco Clemencic / lhcb-core-soft@cernNOSPAMPLEASE.ch
>
>
We are no longer using SVN. It is time to git! Please read everything and please follow the workflow Anatoly describes. If you have any questions email Anatoly and cc the development mailing list. If Anatoly cannot answer them, email Jordi, and if all else fails then Marco Clemencic / lhcb-core-soft@cernNOSPAMPLEASE.ch
 

Standard documentation and other git stuff

Line: 48 to 48
  The main consequences are the following:
Changed:
<
<
  • Packages don't have tags anymore. From now on, a Panoptes project tag affects all its packages. What used to be version v2r26 of Rich/RichOnlineMonitors, now it's the Rich/RichOnlineMonitors that comes with Panoptes v5r6.
    <!-- This is also true even if you tag a package in svn: no package version information will be propagated to the gitlab repository, anyway, so it will not be used when the project is released. -->
>
>
  • CmakeLists and cmt/requirements still need to have a package version number, these should be updated when necessary.
  • However, packages don't have version tags anymore. From now on, a Panoptes project tag affects all its packages. What used to be "version v2r26 of Rich/RichOnlineMonitors" is now the "Rich/RichOnlineMonitors that comes with Panoptes v5r6."
    <!-- This is also true even if you tag a package in svn: no package version information will be propagated to the gitlab repository, anyway, so it will not be used when the project is released. -->
 
  • The tag collector becomes obsolete from the point of view of Panoptes.
Changed:
<
<
  • Since we still need to release Panoptes versions, all packages should at least compile when they are added to the master branch of the git repository. For this reason, it is important to have meaningful development branches for stuff that is not yet stable, which will be merged once the new features are fully implemented and tested.
>
>
  • Since we still need to release Panoptes versions, all packages should at least compile when they are added to the master branch of the git repository. For this reason, it is important to have meaningful development branches for stuff that is not yet stable, which will be merged once the new features are fully implemented and tested.
  The recommended workflow to add features to packages is the following:
Line: 154 to 155
  new shell ( e.g. ssh   -Y   asolomin@lxplusNOSPAMPLEASE.cern.ch )
Changed:
<
<
Only once and forever you configure:
>
>
Only once and forever:
 
  • git config --global user.name "Anatoly Solomin"
  • git config --global user.email anatoly.solomin@cern.ch
Deleted:
<
<
See Prerequisites for more detail.
 
Changed:
<
<
setenv   User_release_area /afs/cern.ch/user/a/asolomin/public/rich_align
>
>
setenv   User_release_area /afs/cern.ch/user/a/asolomin/public/rich_align (the choice of User_release_area is completely up to you, actually)
 cd   $User_release_area
Changed:
<
<
This you do only once, when you start from scratch:
>
>
This you do only once, when you start from scratch (though peform it separately when starting from each new Panoptes release):
 
  • lb-dev   --nightly=lhcb-head   Panoptes   v6r0

cd   $User_release_area/PanoptesDev_v6r0
git   lb-use Panoptes

Changed:
<
<
git   lb-checkout   Panoptes/master   Rich/RichMirrAlign
>
>
not sure yet if you need to keep redoing this; procedure replicating "svn update" is still being established
  • git   lb-checkout   Panoptes/master   Rich/RichMirrAlign
 
<!-- git   commit   -m 'imported Rich/RichMirrAlign from Panoptes' -->
Changed:
<
<
edit Rich/RichMirrAlign/src/RichMirrAlign* &
edit Rich/RichMirrAlign/doc/release_notes.html Rich/RichMirrAlign/CMakeLists.txt Rich/RichMirrAlign/cmt/requirements &
make -j 6
git   add   Rich/RichMirrAlign/CMakeLists.txt
git   add   Rich/RichMirrAlign/cmt/requirements
git   add   Rich/RichMirrAlign/doc/release_notes.html
git   add   Rich/RichMirrAlign/src/RichMirrAlign.cpp
git   add   Rich/RichMirrAlign/src/RichMirrAlignFcn.cpp
git   add   Rich/RichMirrAlign/src/RichMirrAlignFcn.h
>
>
Example (if you're editing code you would obviously not necessarily edit the same files, and of course use your editor of choice)
  • edit Rich/RichMirrAlign/src/RichMirrAlign* &
  • edit Rich/RichMirrAlign/doc/release_notes.html Rich/RichMirrAlign/CMakeLists.txt Rich/RichMirrAlign/cmt/requirements &
  • make -j 6 (check that your code compiles!)

For everything that changed and any new files use git add (e.g. for the previous example):

  • git   add   Rich/RichMirrAlign/src/RichMirrAlign.cpp
  • git   add   Rich/RichMirrAlign/src/RichMirrAlignFcn.cpp
  • git   add   Rich/RichMirrAlign/src/RichMirrAlignFcn.h
  • git   add   Rich/RichMirrAlign/doc/release_notes.html (Please always update the release notes!!!)
  • git   add   Rich/RichMirrAlign/CMakeLists.txt
  • git   add   Rich/RichMirrAlign/cmt/requirements
 
Changed:
<
<
git commit -m 'regularizationMode introduced.'
>
>
Then make sure to update the commit message:
  • git commit -m 'regularizationMode introduced.'
  If after your first commit, you want to edit or expand your commit description before a push use:
  • git commit --amend
Line: 205 to 211
 cd   $User_release_area/PanoptesDev_v6r0
make   install
Changed:
<
<
cd   /afs/cern.ch/user/a/asolomin/public/rich_align_test
>
>
cd   /afs/cern.ch/user/a/asolomin/public/rich_align_test (or wherever your test is)
  lb-run   PanoptesDev v6r0   RichMirrAlign.exe Rich1MirrAlign_Mp6Wi8.0Fm5Mm0Sm0_online_Collision15_i0.conf

Revision 42016-06-17 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 157 to 157
 Only once and forever you configure:
  • git config --global user.name "Anatoly Solomin"
  • git config --global user.email anatoly.solomin@cern.ch
Added:
>
>
See Prerequisites for more detail.
  setenv   User_release_area /afs/cern.ch/user/a/asolomin/public/rich_align
cd   $User_release_area
Line: 168 to 169
 git   lb-use Panoptes

git   lb-checkout   Panoptes/master   Rich/RichMirrAlign

Changed:
<
<
git   commit   -m 'imported Rich/RichMirrAlign from Panoptes'
>
>
<!-- git   commit   -m 'imported Rich/RichMirrAlign from Panoptes' -->
  edit Rich/RichMirrAlign/src/RichMirrAlign* &
edit Rich/RichMirrAlign/doc/release_notes.html Rich/RichMirrAlign/CMakeLists.txt Rich/RichMirrAlign/cmt/requirements &
Line: 182 to 183
  git commit -m 'regularizationMode introduced.'
Changed:
<
<
This will push your changes to the mirralign23 branch.
  • git lb-push Panoptes mirralign23
>
>
If after your first commit, you want to edit or expand your commit description before a push use:
  • git commit --amend
 
Changed:
<
<
Then, submit a merge request from the Panoptes gitlab page

Important: if you noticed a mistake and immediately want to commit a corrected version, you should

  • git lb-push Panoptes mirralign24 (i.e. any different name)
>
>
You will now give your branch a name. At this stage you may choose anything (but pick something unique that helps you keep track of things). Say you pick mirralign23Anatoly.
This will push your changes to the mirralign23Anatoly branch:
  • git lb-push Panoptes mirralign23Anatoly
Then, submit a merge request for mirralign23Anatoly from the Panoptes gitlab page NOTE: Your pushed branch (that contains your commit) MUST COMPILE to get accepted.

Important: if you noticed a mistake and immediately want to commit a corrected version, you should

  • git lb-push Panoptes mirralign24Anatoly (i.e. any different name)
Then, submit a merge request for mirralign24Anatoly, instead of mirralign23Anatoly
 

To run

Revision 32016-06-02 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 10 to 10
 

git

Changed:
<
<
We are no longer using SVN. It is time to git! Please read everything and please follow the workflow Jordi describes. If you have any questions email Jordi and cc the development mailing list.
>
>
We are no longer using SVN. It is time to git! Please read everything and please follow the workflow Anatoly describes. If you have any questions email Anatoly and cc the development mailing list. If Anatoly cannot answer them, email Jordi, and if all else fails then Marco Clemencic / lhcb-core-soft@cernNOSPAMPLEASE.ch
 

Standard documentation and other git stuff

Don't know if any of these are useful yet. Please read the sections below and then fill in information gaps with one of the references.

Changed:
<
<
  • References
>
>
 
Changed:
<
<
  • Places I heard people who actually know how to use git hang out
>
>
  • Places where people who actually know how to use git hang out at:
 
Line: 46 to 48
  The main consequences are the following:
Changed:
<
<
  • Packages don't have tags anymore. From now on, a Panoptes project tag affects all its packages. What used to be version v2r26 of Rich/RichOnlineMonitors, now it's the Rich/RichOnlineMonitors that comes with Panoptes v5r6. This is also true even if you tag a package in svn: no package version information will be propagated to the gitlab repository, anyway, so it will not be used when the project is released.
>
>
  • Packages don't have tags anymore. From now on, a Panoptes project tag affects all its packages. What used to be version v2r26 of Rich/RichOnlineMonitors, now it's the Rich/RichOnlineMonitors that comes with Panoptes v5r6.
    <!-- This is also true even if you tag a package in svn: no package version information will be propagated to the gitlab repository, anyway, so it will not be used when the project is released. -->
 
  • The tag collector becomes obsolete from the point of view of Panoptes.
  • Since we still need to release Panoptes versions, all packages should at least compile when they are added to the master branch of the git repository. For this reason, it is important to have meaningful development branches for stuff that is not yet stable, which will be merged once the new features are fully implemented and tested.
Line: 60 to 62
 
<!-- You may have figured out a way to cheat with the repositories: if you commit your changes to svn, the branch that will be automatically created in gitlab will be merged anyway, without having to go through a merge request. This may sound like a nice workaround to not having to change the way you work, but it actually means breaking the workflow through svn and maybe introducing unwanted changes to the master branch, which is not for development as the svn trunk used to be. This is not intended behavior and, for this reason, after some short coexistence time, the plan is to make the svn repo read-only in the short term. Please report any inconveniences that this may cause you before this happens. We all have our scripts in place, we're not aware of the scripts that other people use and, if they may break, it's better to not find out by surprise. -->

Please get used to the git workflow asap.

Changed:
<
<
>
>
<!--
  You can use the git commands introduced by Marco some time ago. As an example:

lb-dev Panoptes v5r6

Line: 70 to 72
  [... work on Rich/Panoptes, and similarly for any other Panoptes package ...] git lb-push Panoptes branch-name [... submit a merge request ...]
Added:
>
>
-->
 
Added:
>
>
<!--
 

Paras's Email back to Jordi

Hi Jordi (and removing lhcb-rich for now),

Line: 89 to 93
  P.S. OK I am cheating on my promise to go learn here but I am too curious… do merge requests need to be approved, or do they just happen (like commits)? If they need to be approved, who approves them? P.P.S. Thanks for starting the JIRA, we probably should have used it long ago for the mirror alignment rather than keep track of everything on our TWiki. We will try to migrate our open issues to JIRA soon.
Added:
>
>
-->
 
Changed:
<
<

Further Instructions from Jordi, based on my questions

1. About the branches and the svn trunk, there are subtleties. What you say is mostly right, but you should not associate the master branch with the svn trunk. In svn, tagged packages were expected to work, and committers could add unstable stuff to the trunk. With the new workflow that we believe is reasonable for Panoptes, the master branch is expected to always be stable, as opposed to the svn trunk. Unstable contributions to git should take place in branches. So, to summarize, in svn, new contributions go from an unstable trunk to stable branches and from there to release. In git, new contributions go from unstable branches to stable master and from there to release.

>
>

Further Instructions from Jordi, based on Paras's initial questions

 
Added:
>
>
1. You should not associate the master branch with the SVN trunk. In SVN, tagged packages were expected to work, and committers could add unstable stuff to the trunk. With the new workflow that we believe is reasonable for Panoptes, the master branch is expected to always be stable, as opposed to the SVN trunk. Unstable contributions to git should take place in branches. So, to summarize, in svn, new contributions go from an unstable trunk to stable branches and from there to release. In git, new contributions go from unstable branches to stable master and from there to release.
 
Changed:
<
<
2. The svn workflow that you describe, I think can be translated into git like this:
>
>
2. An example git workflow :
 
Changed:
<
<
lb-dev Panoptes v5r6
>
>
lb-dev Panoptes v5r6
 
Changed:
<
<
This creates a local git repository in PanoptesDev_v5r6. This is called a "satellite project". Then, do
>
>
This creates a local git repository in PanoptesDev_v5r6. This is called a "satellite project". Then,
 
Changed:
<
<
cd PanoptesDev_v5r6 git lb-use Panoptes
>
>
cd PanoptesDev_v5r6
git lb-use Panoptes
  This adds the gitlab remote to the local repository of the satellite project, and fetches that remote (makes a local copy of the remote repository, although you still don't see it in your working tree).

The output of this last command will give you information like this:

Changed:
<
<
* [new branch] master -> Panoptes/master * [new tag] v1r0 -> Panoptes/v1r0 [...] * [new tag] v5r5 -> Panoptes/v5r5
>
>
* [new branch] master -> Panoptes/master
* [new tag] v1r0 -> Panoptes/v1r0
[...]
* [new tag] v5r5 -> Panoptes/v5r5
  * [new tag] v5r6 -> Panoptes/v5r6

You will see that there are several references that you can use. Some of them are branches (right now only master) and some of them are tags. You will refer to them as Panoptes/[reference], as in the last column of the previous output.

Lets say you want to develop on top of the Rich/RichMirrorAlignmentOnline package that you can find on the master branch. You would do

Changed:
<
<
git lb-checkout Panoptes/master Rich/RichMirrorAlignmentOnline
>
>
git lb-checkout Panoptes/master Rich/RichMirrorAlignmentOnline
  This is the equivalent of checking out the package from the head. Then you would add new features to the mirror alignment software, and maybe you will commit unfinished work that you want to share with others. In this case, you would do
Changed:
<
<
git lb-push Panoptes newfeature
>
>
git lb-push Panoptes newfeature
  Here, newfeature is the name of your development branch, and you will be able to see it in gitlab. It should be short, but descriptive. If someone else has lb-pushed a new feature branch to Panoptes, you can bring it to your local copy with
Changed:
<
<
git fetch Panoptes git lb-checkout Panoptes/that-branch Rich/RichMirrorAlignmentOnline
>
>
git fetch Panoptes
git lb-checkout Panoptes/that-branch Rich/RichMirrorAlignmentOnline
  You can also tag specific commits on the branch, so you can easily refer to them with others. Please don't use v-tags: the Panoptes project should be the only thing that is v-tagged. If v-tags help you understand better where you are, please prefix them with some short context, e.g. align-v1r0.
Line: 131 to 135
  You can also tag specific commits on the branch, so you can easily refer to them with others. Please don't use v-tags: the Panoptes project should be the only thing that is v-tagged. If v-tags help you understand better where you are, please prefix them with some short context, e.g. align-v1r0.
Deleted:
<
<
  3. I think the lb-use and lb-checkout commands also solve the case where you want to use packages as plugins from other projects. You can very consistently do
Changed:
<
<
git lb-use DaVinci git lb-checkout DaVinci/v38r1 Phys/DaVinci
>
>
git lb-use DaVinci
git lb-checkout DaVinci/v38r1 Phys/DaVinci
  So this is how you would use the Phys/DaVinci package in Panoptes (regardless of the fact that it would not make any sense).
Line: 139 to 142
  So this is how you would use the Phys/DaVinci package in Panoptes (regardless of the fact that it would not make any sense).
Deleted:
<
<
  4. About releasing a new version of Panoptes every time a package has a substantial improvement: I don't have a correct answer to this question. New Panoptes releases are created when there's interest in them. It may be for several reasons: to have new features released, to use a new release of Online, to compile with a new CMTCONFIG,... I never oppose creating a new release if it is requested.
Added:
>
>
5. Who approves merge requests?
<!--Right now, in the period of coexistence, they are automatically approved by the librarian account, liblhcb. -->
When the svn repository becomes read-only, it will make sense to designate who should be allowed to accept them. [Note: still don't know who this is]

Anatoly's example git workflow

Anatoly was kind enough to provide us with his git workflow so far. He is still learning as are we, but this will certainly help you get started.

To build

new shell ( e.g. ssh   -Y   asolomin@lxplusNOSPAMPLEASE.cern.ch )

Only once and forever you configure:

  • git config --global user.name "Anatoly Solomin"
  • git config --global user.email anatoly.solomin@cern.ch

setenv   User_release_area /afs/cern.ch/user/a/asolomin/public/rich_align
cd   $User_release_area

This you do only once, when you start from scratch:

  • lb-dev   --nightly=lhcb-head   Panoptes   v6r0

cd   $User_release_area/PanoptesDev_v6r0
git   lb-use Panoptes

git   lb-checkout   Panoptes/master   Rich/RichMirrAlign
git   commit   -m 'imported Rich/RichMirrAlign from Panoptes'

edit Rich/RichMirrAlign/src/RichMirrAlign* &
edit Rich/RichMirrAlign/doc/release_notes.html Rich/RichMirrAlign/CMakeLists.txt Rich/RichMirrAlign/cmt/requirements &
make -j 6
git   add   Rich/RichMirrAlign/CMakeLists.txt
git   add   Rich/RichMirrAlign/cmt/requirements
git   add   Rich/RichMirrAlign/doc/release_notes.html
git   add   Rich/RichMirrAlign/src/RichMirrAlign.cpp
git   add   Rich/RichMirrAlign/src/RichMirrAlignFcn.cpp
git   add   Rich/RichMirrAlign/src/RichMirrAlignFcn.h

git commit -m 'regularizationMode introduced.'

This will push your changes to the mirralign23 branch.

  • git lb-push Panoptes mirralign23

Then, submit a merge request from the Panoptes gitlab page

Important: if you noticed a mistake and immediately want to commit a corrected version, you should

  • git lb-push Panoptes mirralign24 (i.e. any different name)

To run

new shell ( e.g. ssh   -Y   asolomin@lxplusNOSPAMPLEASE.cern.ch )

setenv   User_release_area /afs/cern.ch/user/a/asolomin/public/rich_align

cd   $User_release_area/PanoptesDev_v6r0
make   install

cd   /afs/cern.ch/user/a/asolomin/public/rich_align_test

lb-run   PanoptesDev v6r0   RichMirrAlign.exe Rich1MirrAlign_Mp6Wi8.0Fm5Mm0Sm0_online_Collision15_i0.conf

List of questions to be asked and answered (by Jordi, etc...) at an appropriate time

What is the reason to commit (but without a push) the fact that a package was checked out?

Jordi talked about us being able to create some sort of “local” tag, like “mirralign-RichMirrAlign-v19r2”. How do we do that, as the unique names given to every branch [such as "mirralign23"] exist only between your push and the moment when Jordi (with approval from Chris) "merges" it to the master. (Of course, Jordi does the merge only after you "asked" for it online.)

 
Changed:
<
<
5. Who approves merge requests? Right now, in the period of coexistence, they are automatically approved by the librarian account, liblhcb. When the svn repository becomes read-only, it will make sense to designate who should be allowed to accept them.
>
>
Also, is there a way, say after you commit “mirralign23”, to do something similar to “svn update” to get your local branch to be in sync with the master branch (say, if your changes are accepted, then someone else changes it), or would you have to check out the head version each time?
 
Added:
>
>
Questions on which Anatoly was not 100% sure:
  • Q: Git add must be done immediately before a git commit/lb-push and if any file changes git add must be done again?
    • A: not exactly: "adding" is just formation of the list of files to be gitted, so you can do that any time. However, I'm not 100% sure, so please keep this Q for the experts...
  • Q: You cannot "git lb-push" to a branch already existing in the remote repository, unless you checked out from that specific branch?
    • A: I think, it is without "unless", meaning that such a branch is always unique, but keep this Q too.
  • Q: How during running one could use another project’s package ( before this was possible by getpacking whatever package you want to use, then using e.g. SetupProject Panoptes vXrY --use Rich/RichAlignment ). This is useful, for example, if we work on Rich/RichAlignment but a new version of Rec has not yet been released. Something to figure out later though as hopefully this is a rare case.
    • A: No clue. But of course, our use case is a "multiproject" one (Panoptes + Brunel), and in some situations (e.g. in the Ganga case) we need a recipe for that.
 
Changed:
<
<

The JIRA rich project. (we are not using JIRA yet, but we really should consider it)

>
>

The JIRA rich project.

 
Changed:
<
<
Jordi has created a jira project, which you can find here. Jordi is not an expert in jira, so Jordi cannot guarantee that the configuration that he has done is ideal. If you know more than Jordi about project management (which is very likely), please help create a nice framework to handle rich issues.
>
>
Jordi has created a jira project, which you can find here. Jordi is not an expert in jira, so Jordi cannot guarantee that the configuration that he has done is ideal. If you know more than Jordi about project management (which is very likely), please help create a nice framework to handle rich issues.
  The jira group does NOT impose any change in the workflow. If you prefer to keep managing issues by email, or if you use an external service, or even if you keep track of your tasks by tweeting them, this is still fine. However, people who have used jira have a good experience in their opinion, so it may be worth giving it a try.
Line: 156 to 230
  Rich jira issues have references like RICH-XXXX, where XXXX is a number. If you mention a jira issue in a git commit, jira will automatically create a comment in its corresponding issue page.
Changed:
<
<
Members of lhcb-rich-software have "developers" access to the rich jira project. Let me know if you prefer that Jordi changes it to lhcb-rich.
>
>
Members of lhcb-rich-software have "developers" access to the rich jira project. Let Jordi know if you prefer that Jordi changes it to lhcb-rich.
 

The End

Revision 22016-04-14 - ParasNaik

Line: 1 to 1
 
META TOPICPARENT name="LHCbRichMirrorAlign"
Line: 10 to 10
 

git

Changed:
<
<
We are no longer using SVN. It is time to git!
>
>
We are no longer using SVN. It is time to git! Please read everything and please follow the workflow Jordi describes. If you have any questions email Jordi and cc the development mailing list.
 

Standard documentation and other git stuff

Revision 12016-04-12 - ParasNaik

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="LHCbRichMirrorAlign"

git instructions for the RICH mirror alignment

Table of Contents

git

We are no longer using SVN. It is time to git!

Standard documentation and other git stuff

Don't know if any of these are useful yet. Please read the sections below and then fill in information gaps with one of the references.

Transition of Panoptes from svn to git.

The Panoptes project has been moved to gitlab. https://gitlab.cern.ch/LHCb-SVN-mirrors/Panoptes

In the cloned gitlab repository, all contributions to svn up to release v5r5 have been getpacked and committed, but no other history is kept about them. After release v5r5, all the history has been kept.

For reference, Jordi has kept a complete copy of the svn repository at https://gitlab.cern.ch/lhcb-rich/panoptes In this longer version, all the svn history is kept and, additionally, all tagged releases have been getpacked and attached at their corresponding point in history. This is not intended to be used in practice, just for reference. Jordi will update this repository just once in a while but not very frequently.

Jordi has created the gitlab group lhcb-rich, intended for rich related repositories. https://gitlab.cern.ch/groups/lhcb-rich The lhcb-rich-software egroup is linked to the gitlab group such that all members have "developer" access.

<!--    The gitlab and svn repositories are synced every 30 minutes, in both directions: 
      if you commit something to svn, it will show up in the gitlab repository, and accepted merge requests in gitlab will appear in svn. -->

There is a major change in the workflow, since we cannot use tagged packages in git anymore.

The main consequences are the following:

  • Packages don't have tags anymore. From now on, a Panoptes project tag affects all its packages. What used to be version v2r26 of Rich/RichOnlineMonitors, now it's the Rich/RichOnlineMonitors that comes with Panoptes v5r6. This is also true even if you tag a package in svn: no package version information will be propagated to the gitlab repository, anyway, so it will not be used when the project is released.
  • The tag collector becomes obsolete from the point of view of Panoptes.
  • Since we still need to release Panoptes versions, all packages should at least compile when they are added to the master branch of the git repository. For this reason, it is important to have meaningful development branches for stuff that is not yet stable, which will be merged once the new features are fully implemented and tested.

The recommended workflow to add features to packages is the following:

  • Implement the features in a branch.
  • Test them.
  • Push the branch to an upstream repository (e.g. a fork of the project in gitlab).
  • Submit a merge request, so the new features are included in the master branch.

<!-- You may have figured out a way to cheat with the repositories: if you commit your changes to svn, the branch that will be automatically created in gitlab will be merged anyway, without having to go through a merge request. This may sound like a nice workaround to not having to change the way you work, but it actually means breaking the workflow through svn and maybe introducing unwanted changes to the master branch, which is not for development as the svn trunk used to be. This is not intended behavior and, for this reason, after some short coexistence time, the plan is to make the svn repo read-only in the short term. Please report any inconveniences that this may cause you before this happens. We all have our scripts in place, we're not aware of the scripts that other people use and, if they may break, it's better to not find out by surprise. -->

Please get used to the git workflow asap.

You can use the git commands introduced by Marco some time ago. As an example:

lb-dev Panoptes v5r6 cd PanoptesDev_v5r6 git lb-use Panoptes git lb-checkout Panoptes/v5r6 Rich/Panoptes [... work on Rich/Panoptes, and similarly for any other Panoptes package ...] git lb-push Panoptes branch-name [... submit a merge request ...]

Paras's Email back to Jordi

Hi Jordi (and removing lhcb-rich for now),

Congratulations on getting this all into git! I will take your advice to start using it right away, but since I have not used git I cannot predict anything that I am about to do wrong smile At the moment I am a little lost about development branches and master branches. If the master branch is the trunk, development branches are the local copies, and merge requests are commits, then I get it already, but if I have that wrong or there are subtleties then I will learn.

Speaking only to our experience in the mirror alignment (but maybe others share this), we are used to checking out the package head locally using svn, checking for svn updates often to avoid falling behind on each other’s changes, committing often (only when code compiles of course), updating the release notes, and and tagging code as a new version when appropriate. So if these habits are still the right ones for packages in git (except maybe the tagging), we will probably be fine.

The ability to have our packages in project Panoptes, but be able to be used as plug-ins to other projects (e.g. AlignmentOnline), has worked very well for us. Keeping track of major improvements via package tags and the tag collector thus has been a nice way to keep track of major improvements; and we would know for certain that only a tagged package would go into any new version of the project created by the project manager.

Since we are moving from tagged packages and projects to tagged projects only, would it make sense to then release a new version of Panoptes every time a package of Panoptes has a substantial improvement? If this is getting carried away, then how else should we internally keep track of our package “tags”? For example we could use svn revision number, but I suppose git will have its own revision numbers too?

Cheers,

~ Paras

P.S. OK I am cheating on my promise to go learn here but I am too curious… do merge requests need to be approved, or do they just happen (like commits)? If they need to be approved, who approves them? P.P.S. Thanks for starting the JIRA, we probably should have used it long ago for the mirror alignment rather than keep track of everything on our TWiki. We will try to migrate our open issues to JIRA soon.

Further Instructions from Jordi, based on my questions

1. About the branches and the svn trunk, there are subtleties. What you say is mostly right, but you should not associate the master branch with the svn trunk. In svn, tagged packages were expected to work, and committers could add unstable stuff to the trunk. With the new workflow that we believe is reasonable for Panoptes, the master branch is expected to always be stable, as opposed to the svn trunk. Unstable contributions to git should take place in branches. So, to summarize, in svn, new contributions go from an unstable trunk to stable branches and from there to release. In git, new contributions go from unstable branches to stable master and from there to release.

2. The svn workflow that you describe, I think can be translated into git like this:

lb-dev Panoptes v5r6

This creates a local git repository in PanoptesDev_v5r6. This is called a "satellite project". Then, do

cd PanoptesDev_v5r6 git lb-use Panoptes

This adds the gitlab remote to the local repository of the satellite project, and fetches that remote (makes a local copy of the remote repository, although you still don't see it in your working tree).

The output of this last command will give you information like this:

* [new branch] master -> Panoptes/master * [new tag] v1r0 -> Panoptes/v1r0 [...] * [new tag] v5r5 -> Panoptes/v5r5 * [new tag] v5r6 -> Panoptes/v5r6

You will see that there are several references that you can use. Some of them are branches (right now only master) and some of them are tags. You will refer to them as Panoptes/[reference], as in the last column of the previous output.

Lets say you want to develop on top of the Rich/RichMirrorAlignmentOnline package that you can find on the master branch. You would do

git lb-checkout Panoptes/master Rich/RichMirrorAlignmentOnline

This is the equivalent of checking out the package from the head. Then you would add new features to the mirror alignment software, and maybe you will commit unfinished work that you want to share with others. In this case, you would do

git lb-push Panoptes newfeature

Here, newfeature is the name of your development branch, and you will be able to see it in gitlab. It should be short, but descriptive. If someone else has lb-pushed a new feature branch to Panoptes, you can bring it to your local copy with

git fetch Panoptes git lb-checkout Panoptes/that-branch Rich/RichMirrorAlignmentOnline

You can also tag specific commits on the branch, so you can easily refer to them with others. Please don't use v-tags: the Panoptes project should be the only thing that is v-tagged. If v-tags help you understand better where you are, please prefix them with some short context, e.g. align-v1r0.

3. I think the lb-use and lb-checkout commands also solve the case where you want to use packages as plugins from other projects. You can very consistently do

git lb-use DaVinci git lb-checkout DaVinci/v38r1 Phys/DaVinci

So this is how you would use the Phys/DaVinci package in Panoptes (regardless of the fact that it would not make any sense).

4. About releasing a new version of Panoptes every time a package has a substantial improvement: I don't have a correct answer to this question. New Panoptes releases are created when there's interest in them. It may be for several reasons: to have new features released, to use a new release of Online, to compile with a new CMTCONFIG,... I never oppose creating a new release if it is requested.

5. Who approves merge requests? Right now, in the period of coexistence, they are automatically approved by the librarian account, liblhcb. When the svn repository becomes read-only, it will make sense to designate who should be allowed to accept them.

The JIRA rich project. (we are not using JIRA yet, but we really should consider it)

Jordi has created a jira project, which you can find here. Jordi is not an expert in jira, so Jordi cannot guarantee that the configuration that he has done is ideal. If you know more than Jordi about project management (which is very likely), please help create a nice framework to handle rich issues.

The jira group does NOT impose any change in the workflow. If you prefer to keep managing issues by email, or if you use an external service, or even if you keep track of your tasks by tweeting them, this is still fine. However, people who have used jira have a good experience in their opinion, so it may be worth giving it a try.

In jira there are these things called "components", which correspond to different unrelated topics (or mostly unrelated) within a project. I have created four software components: monitoring (for Panoptes issues), alignment (for mirror alignment), piquet (for piquet software tools) and reconstruction (for reconstruction software). In principle, jira can be useful for hardware projects as well, so feel free to create as many components as you want for hardware and for other software topics.

Rich jira issues have references like RICH-XXXX, where XXXX is a number. If you mention a jira issue in a git commit, jira will automatically create a comment in its corresponding issue page.

Members of lhcb-rich-software have "developers" access to the rich jira project. Let me know if you prefer that Jordi changes it to lhcb-rich.

The End

-- ParasNaik - 2016-04-12

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback