CMSPublic Web>SWGuide>SWGuideDocumentationPlan (2009-07-17, PhilipAllfrey)

EditAttachPDF

CMS Software Documentation Plan

Complete:

Contacts

Convener: Kati Lassila-Perini
Group contacts: SWGuideDocumentationContacts
e-mail list: cms-doc-contacts@cernNOSPAMPLEASE.ch

Goal of the page

This page drafts the concrete steps to be taken towards completion of the CMS SW documentation.

Introduction

A documentation campaign has been started to make sure that all relevant aspects of the CMS Software are documented. The work is based of the CMS Software documentation policy.

The goals of the campaign are:

to make sure that the existing documentation appears properly named and linked in the CMS Offline Guide
to make clear the distinction between
- "How to use", e.g.
  - how to read and analyse the standard data
  - how to do a private MC production with a specific generator (or generator settings) using standard configuration files for simulation and reconstruction.
  - This part of the documentation should reside in the WorkBook (and be linked to the Offline Guide).
- "What has been done"
  - the documentation of the standard software procedures that have been applied to produce the standard data.
  - This part of the documentation should reside in the Offline Guide.
to identify the missing pieces in terms of the tasks that the users need to perform.

Contact persons

The user support and documentation contact persons from physics, offline and computing groups are responsible for the main pages of their area in the Offline Guide.

They organize the documentation work within the group with technical help of the User Support. They also help identifying experts in the group in case of user questions on areas under the group’s responsibility

The contact persons are listed in SWGuideDocumentationContacts.

Step 1: Identify the existing documentation

Identify the existing documentation concerning their group (WorkBook, Offline Guide, other twiki/html pages, reference manual domain pages). All of it should be linked to the Offline Guide, for many areas this is already the case.

If your SWGuide main page is empty, start with linking the existing documentation to the Offline Guide main page of your area. All twiki pages about CMSSW (i.e. containing scramv1 project CMSSW CMSSW_nnn cvs co ....) should be renamed so that their names start with a string "SWGuide" (if you need help in renaming pages, contact me).

To find the pages: show details hide details To find any documentation twiki pages outside WorkBook and Offline Guide, have a look at this list of pages, this is the result of a search of CMS twiki pages containing "scramv1 project CMSSW" and check if there are pages connected to your area outside WorkBook and Offline guide (i.e. the names not starting with "WorkBook" and "SWGuide").

Step 2: Standard sequences

This part concentrates on the "What has been done" aspect of the Offline Guide documentation. The goal is to make sure that all the algorithms and procedures (triggered by the standard sequences in the default configuration files) when processing the standard data are documented.

Reference documentation

The standard sequences for simulation and reconstruction in Configuration/StandardSequences should all be documented.

These sequences are listed in the reference manual and this page is linked to SWGuideReco.

All sequences, such as trackerlocalreco should be linked to the location in the Offline Guide where the full documentation will be given, so that a reader will be able to find the documentation for each sequence by following the link. These links will be completed during the documentation procedure.

See the latest modifications for the standard sequences page show details hide details If you update the page (i.e. modify the file Configuration/StandardSequences/doc/Sequences.doc) or if you want to see the very latest modifications in cvs head, do the following

   cd ~/scratch1
   scramv1 project CMSSW CMSSW_2_1_0_pre4
   cd CMSSW_2_1_0_pre4/src
   project CMSSW
   cvs co Configuration/StandardSequences/doc
   cvs co Documentation/ReferenceManualPages
   cvs co Documentation/ReferenceManualScripts 
   scramv1 b referencemanual  
   firefox ../doc/html/index.html

and follow the link Standard sequences for configuration files.

Responsibilities

Each group is responsible for providing the documentation for the procedures and algorithms triggered by the standard sequences in their area.

The detailed mapping of responsibilities: show details hide details

sequence reconstruction = {localreco, globalreco_plusRS, highlevelreco_plusRS}

localreco = {trackerlocalreco, muonlocalreco, calolocalreco}:

calolocalreco = {ecalLocalRecoSequence & hcalLocalRecoSequence}

trackelocalreco: Tracker DPG
muonlocalreco: Muon DPG
ecalLocalRecoSequence: ECAL DPG
hcalLocalRecoSequence: HCAL DPG

sequence globalreco = {ckftracks, ecalClusters, caloTowersRec, recoJets, metreco,muonreco_plus_isolation }

sequence globalreco_plusRS = {globalreco, rstracks }

ckftracks: Tracker DPG
ecalClusters: ECAL DPG
caloTowersRec: HCAL DPG
recoJets: jetmet POG
metreco: jetmet POG
muonreco_plus_isolation: muon POG
rstracks: Tracker DPG

sequence highlevelreco = {offlineBeamSpot, recopixelvertexing, vertexreco, recoJetAssociations, btagging, tautagging, egammareco,particleFlowReco,PFTau}

sequence highlevelreco_plusRS = {highlevelreco,vertexreco_rs}

offlineBeamSpot: Tracker DPG
recopixelvertexing: Tracker DPG
vertexreco: Tracker DPG and btagging POG
recoJetAssociations: jetmet POG
btagging: btagging POG
tautagging: PF/tau POG
egammareco: egamma POG
particleFlowReco: PF/tau POG
PFTau: PF/tau POG
vertexreco_rs: Tracker DPG

sequence simulation = {psim,pdigi,genParticleCandidates}

genParticleCandidates: Physics Tool offline group

sequence psim = { VtxSmeared, g4SimHits }

VtxSmeared: Simulation offline group
g4SimHits: Simulation offline group

sequence pdigi = { mix,doAllDigi, trackingParticles}

mix: Simulation offline group
trackingParticles: tracker PDG

sequence doAllDigi = { trDigi & calDigi & muonDigi }

trDigi: tracker PDG
calDigi: ECAL/HCAL PDG
muonDigi: Muon PDG

How to proceed?

General

List all algorithms and procedures triggered by the standard sequences assigned to your group.

Note that much of the requested material may already exist, and it may just be needed to link it properly - and in many cases it is even already linked.
If an up-to-date note on the subject exists, it is enough to have a link to this note (and eventually a comment page for updates to it) .

Make it very clear what part of the documentation refers to the standard sequences, and what are additional options - example in SWGuideTrackReco. Make it clear already at the main page level, it is not enough to mention it only in the documentation page of a specific algorithm.

Make sure that the name of the sequence (if reasonable and stable in time) appears in the page, preferrably in its title.

Documenting different versions

For documenting different versions, use the approach in WorkBookBTagging, show details

hide details

This is a (very good) example for a WorkBook "How to use" page, and not a Offline Guide "What has been done" page, but the way to present information for several versions is valid for both cases.

The instructions for different versions are stored in separate pages the name of which contains the version number (e.g. WorkBookBTagData160, WorkBookBTagData170 etc)
The contents of the default version page can be included to the current instruction page without retyping using
```
%INCLUDE{"WorkBookBTagData160" pattern="(?:.*?){0}.*?(.*?).*"}% 
```
and tagging the contents of the page to be included by having
```
 
 
...
 
```
around the text to be included. In this way, it is easy to change to a new default version in the documentation page, and the documentation page will not be overloaded by instructions for all versions.
It is useful to have an explicit link to the specific version pages, after the included text, as the actual location of the instruction does not appear for the text included by the mechanishm above.

Have a look at this page (use Raw View option from the bottom, to avoid the risk of modifying the page accidentally...)

Documenting the algorithms

Identify a person to write the documentation of each algorithm and define a time-scale.

Short guide-lines for writing:

Think of writing to a new developer who wants to understand what the code is doing.
There is a template page : SWGuideAlgorithmTemplate.
- I can create the template pages, send me a list of pages that are needed.
Do not duplicate information, but cross-link to the doxygen class and package documentation, e.g. TrackRefitter
- More details on doxygen links in SWGuideContributors#Predefined_variables

Eventually, all options and alternative algorithms should be documented, but the steps described here should guarantee that at least the standard procedures are all covered. Feel free to document the rest, if this is already done!

Step 3: Complete "How to use"

Identify the task that user may want to perform

POG/DPG group contacts:

Make sure that adequate tutorials exist for accessing the reconstructed data. For the data in RECO and AOD, the tutorials should reside in the WorkBook, in Chapter 7.

L1/HLT/DQM group contact:

Make sure that any task end-user may want to do is properly documented and clearly distinguished from the rest of the documentation.

PAG group contacts:

Make sure that the SWGuide pages contain the links to your example code and to the instructions.

Progress report

Documentation reviews

The progress will be followed up in reviews organized for each group separately as follows

week 0: Start-up meeting with KLP, convener(s), doc contact
- set up goals
- find internal reviewers (e.g. new-comers in the group)
week 0-2: group makes an effort to have the documentation on the agreed level
week 2-3: review by User support & reviewers
week 4: follow-up meeting to assess the results.

These reviews will start towards the end of 2008.

POG group reports

Offline group reports

PAG group reports

Documentation session 17 June 2008

The agenda: http://indico.cern.ch/conferenceDisplay.py?confId=34255.

The contact persons of each group should prepare a short report on the status of documentation.

Prepare 1-2 slides at maximum

if you cannot be present, let me know and attach your slides (or a text file).

Prepare to show through how the different areas are covered.

DPG/POG contacts:

The Offline Guide (see SWGuide) main page navigation bar has now the following links for each RECO object:

the respective Offline Guide page
data formats
standard sequences
how to access data.

The DPG/POG contacts should make sure and report that:

the links from the Offline Guide navigation bar (see SWGuide) are correct.
all algorithms used by the standard sequences are listed in your Offline Guide page and have a documentation page
- provide a template page, even if the contents are not complete (see example in SWGuideTrackReco).
- find names to fill in the page
no software documentation pages reside outside the Offline Guide or WorkBook.

PAG contacts:

The PAG contacts should make sure that Offline Guide page is up-to-date and gives the following information:

where does the analysis code reside
links to tutorials and instructions
related links to other areas and to the group main page

Reconstruction and simulation contacts:

Check that Configuration/StandardSequences/doc/Sequences.doc is up-to-date for 2_1_0

Other SW group contacts:

Any news since the last meeting (see reports in http://indico.cern.ch/conferenceDisplay.py?confId=28445#2008-04-04)?

Documentation session 04 April 2008

In the Documentation session during the Computing and Offline Workshop each group is asked to prepare a short report on the status of documentation. You should shortly mention:

The progress achieved in your section in the Offline Guide.
The most urgent items to be completed.
Are there still software documentation pages outside the Offline Guide or WorkBook?

The simulation, reconstruction and DPGs should report on progress on documenting the standard sequences.

Prepare 1-2 slides at maximum, if you cannot be present, please attach your slides (or a text file).

Steps 1 and 2: mail to contact persons 09 Jan 2008

The documentation contacts should give a brief, intermediate progress report on the work according to this documentation plan by Wednesday, Jan 23 by answering the few questions below. This concerns the contacts from PDGs and POGs, and simulation and reconstruction offline groups. PAGs are not yet concerned although any feedback from you is most welcome.

The reference manual page about the standard sequences is in cvs head, you should read the updated page as instructed above

To this updated page, I have added links to the placeholders for documentation in different areas in the Offline Guide. These are just initial guesses, you are free to change them when needed, or let me know and I'll do it. I would like you to have a look at the work done by Giuseppe Cerati for the SWGuideTrackReco page. He has clearly indicated the area where the standard procedures will be documented and to start with, he has given explicit links to the appropriate areas in an already existing document. I suppose this can be done for many other areas. I have added a similar area to many Offline pages, if your page does not have one yet, add it.

For the progress report, take a look at the updated standard sequences page (link to the instructions above) and answer to the following questions:

Have you identified the documentation connected to the sequence (example: does the documentation about trackerlocalreco or trdigi exist - see the assignments of sequences to different groups above)?
Have you linked the documentation to the appropriate place in the Offline Guide? Some "appropriate places" are already suggested in the links of the standard sequence page, let me know where to point to for those sequences that do not have a link yet.
Is the documentation up-to-date?
Are the optional features and procedures separated from the default behaviour (i.e. what you get when you look at the standard data versus different options which can be used but are not done by default)?

Note that all the reconstruction sequences do not appear yet in this standard sequence page, I would like Andrea or Shahram to update the reconstruction part of this page to correspond to the most up-to-date Reconstruction.cff. I have not updated the page myself not knowing if the head version is the one to follow and if there are some imminent changes.

SWGuideDocPlanReport23Jan

Review status

Reviewer/Editor and Date (copy from screen)	Comments
KatiLassilaPerini - 29 Jan 2008	updates for cross-linking to ref. manual
KatiLassilaPerini - 9 Jan 2008	updates for the standard sequences page
KatiLassilaPerini - 29 Nov 2007	clarifications from the e-mail exchange in green
KatiLassilaPerini - 23 Nov 2007	created the page