(r84) LHCbDIRACReleaseSchedule < LHCb

LHCb Web>LHCbComputing>LHCbDIRACReleaseSchedule (revision 84) (raw view)~~EditAttachPDF~~
---+++ Releases schedule for LHCbDIRAC

---++ Based on DIRAC v6r12:

---+ LHCbDIRACv8r0

*Features to be included* with link to ITS if available, or with status

Need to be in:
   * Everything in https://its.cern.ch/jira/browse/LHCBDIRAC-292?jql=project%20%3D%20LHCBDIRAC%20AND%20fixVersion%20%3D%20v8r0
   * https://its.cern.ch/jira/browse/LHCBDIRAC-130 (PhC, use of BK from Ganga)
   * https://its.cern.ch/jira/browse/LHCBDIRAC-260 (MC)
   * https://its.cern.ch/jira/browse/LHCBDIRAC-259 (MC)
   * https://its.cern.ch/jira/browse/LHCBDIRAC-258 (MC)
   * https://its.cern.ch/jira/browse/LHCBDIRAC-262
   * https://its.cern.ch/jira/browse/LHCBDIRAC-191 (SetupProject LHCbDIRAC within the pilots, real-pilots only version)

Maybe to be in:
   * https://savannah.cern.ch/task/?17737 (slide 4 of https://indico.cern.ch/getFile.py/access?contribId=25&sessionId=4&resId=0&materialId=slides&confId=236650)
   * Adapt LHCb workflows to use DIRAC Workflows - almost done
   * https://its.cern.ch/jira/browse/LHCBDIRAC-216 (use of new templates) - not done

| *Date* | *Status* |
| 01/08/2014 | features definition freeze |
| 13/08/2014 | First pre-release on certification. trunk freezes: only fixes allowed | 
| //2014 | Last pre-release on certification. Last round of tests launched |
| //2014 | Release is created and installed in production setup |


   *News and changes to be applied by hand*

The SAMSystem disappeared. The last remaining agent of SAMSystem (SAMAgent) has been moved to ResourceStatusSystem.

Agents:
- New TransformationSystem/GridCollector agent, as a complement for indexing productions. This substitutes the old one that was running on volhcb02.
- New TransformationSystem/MCSimulationTestingAgent
- The SiteDirectors are now LHCb specific. Nothing should be done, just check.

On the Transformation DB:

ALTER TABLE Transformations ADD COLUMN Hot BOOLEAN DEFAULT FALSE;

New table:

DROP TABLE IF EXISTS StoredJobDescription;
CREATE TABLE StoredJobDescription(
  TransformationID INTEGER NOT NULL,
  JobDescription LONGBLOB NOT NULL,
  PRIMARY KEY (TransformationID),
  FOREIGN KEY (TransformationID) REFERENCES Transformations (TransformationID)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;




---++ Based on DIRAC v6r11:

---+ LHCbDIRACv7r16

   * Beware of some changes in UserJobFinalization: second replica should be done only on user request, and use directly the RMS

*Features to be included* with link to ITS if available, or with status
Need to be in:

   * https://its.cern.ch/jira/browse/LHCBDIRAC-256 - OK
   * https://its.cern.ch/jira/browse/LHCBDIRAC-242 - OK
   * https://its.cern.ch/jira/browse/LHCBDIRAC-234 - OK
   * https://its.cern.ch/jira/browse/LHCBDIRAC-168 - OK
   * https://its.cern.ch/jira/browse/LHCBDIRAC-169 - OK
   * https://its.cern.ch/jira/browse/LHCBDIRAC-252 - OK
   * https://its.cern.ch/jira/browse/LHCBDIRAC-158 - OK
   * https://its.cern.ch/jira/browse/LHCBDIRAC-264 - OK

| *Date* | *Status* |
| 20/02/2013 | features definition freeze |
| 25/02/2013 | First pre-release on certification. trunk freezes: only fixes allowed | 
| 04/04/2014 | Last pre-release on certification. Last round of tests launched |
| 07/04/2014 | Release is created and installed in production setup |


   *Changes to be applied by hand* (dbod-lbprod)


https://github.com/DIRACGrid/DIRAC/wiki/DIRAC-v6r11#few-changes-to-field-definitions-affecting-several-dbstables

Stop the agent:
   * BkWatchAgent (volhcb20)
And the services:
   * ProductionRequest (lbvobox03)
   * TransformationManager (volhcb19, volhcb24, volhcb29)

Then:

<verbatim>
USE ProductionRequestDB;

ALTER TABLE ProductionRequests ADD RequestWG VARCHAR(128) DEFAULT '' AFTER RequestPDG;
ALTER TABLE ProductionRequests ADD INDEX RequestWG (RequestWG);


USE TransformationDB;

CALL CopyBkQueries;

</verbatim>

Update to version v7r16 and then restart the services and agents.




---+ LHCbDIRACv7r15

Notes:
* For this release we should if we want to use the new packaging (BeautyDIRAC v1r0), now under consideration: https://indico.cern.ch/getFile.py/access?sessionId=2&resId=0&materialId=1&confId=267402
* The new steps will need to define the System Config. In case it is not defined (as it is the case for steps already created), "ANY" will be set

*Changes in DIRAC* to consider:

   * New RMS everywhere
      * Needs replication and deletion tests, using FTS
   * New Workflow Package
	* little changes in LHCbDirac, usual tests for LHCb


*Features to be included* with link to ITS if available, or with status

Need to be in:
   * Adaptable MC jobs
   * https://its.cern.ch/jira/browse/LHCBDIRAC-71
   * MCExtensionAgent: https://its.cern.ch/jira/browse/LHCBDIRAC-141
   * New Productions State Machine machinery
   * New TransformationFiles State Machine machinery: https://its.cern.ch/jira/browse/LHCBDIRAC-192
      * With https://its.cern.ch/jira/browse/LHCBDIRAC-194
   * Space token information in LHCbDIRAC accounting

| *Date* | *Status* |
| 21/10/2013 | features definition freeze |
| 22/10/2013 | First pre-release on certification. trunk freezes: only fixes allowed | 
| 13/01/2014 | Last pre-release on certification. Last round of tests launched |
| 19/01/2014 | Release is created and installed in production setup |

*Changes to be applied by hand*

   * Remove table SLSStorage from ResourceManagementDB
   * install_project: change in CS: move to NoSoftwareInstallation (as it is for certification and development)
   * Change in CS: Add "Idle" to Transformation/Agent/WorkflowTaskAgent/UpdateTaskStatus
   * Change in CS to enable the policies on the CE (copying from certification)


---+ LHCbDIRAC v7r14

Note: this is fast track release. DIRAC v6r9 is based on DIRAC v6r8 and will contain only the new RMS system. LHCbDirac tags will be made starting from the v7r14 branch. *NB: the v7r14 branch will be created starting from the v7r13 branch!*

*Features to be included* with link to ITS if available, or with status

Need to be in:
   * Only a change in the workflow to use the new RMS -> to be completed
   * The LHCbTestDirac will contain some system tests that needs to be run during the certification

| *Date* | *Status* |
| 10/06/2013 | features definition freeze |
| 11/06/2013 | First pre-release on certification. v7r14 branch freezes: only fixes allowed | 
| 20/06/2013 | Last pre-release on certification. Last round of tests launched |
| 24/06/2013 | Release is created and installed in production setup |


---++ Based on DIRAC v6r8:

---+ LHCbDIRAC v7r13

Note: this is fast track release. DIRAC v6r8 is based on DIRAC v6r7 and will contain only fixes, nothing new. LHCbDirac tags will be made starting from the trunk as usual (anyway, there is not much to put it)

*Features to be included* with link to ITS if available, or with status

Need to be in:
   * https://its.cern.ch/jira/browse/LHCBDIRAC-109
   * https://its.cern.ch/jira/browse/LHCBDIRAC-85

| *Date* | *Status* |
| 07/04/2013 | features definition freeze |
| 08/04/2013 | First pre-release on certification. Trunk freezes: only fixes allowed | 
| 11/04/2013 | Last pre-release on certification. Last round of tests launched |
| 15/04/2013 | Release is created and installed in production setup |



---++ Based on DIRAC v6r7:

---+ LHCbDIRAC v7r12

*Features to be included* with link to ITS if available, or with status

Need to be in:
   * Certification procedure second sprint: https://its.cern.ch/jira/secure/RapidBoard.jspa?rapidView=801&view=planning&selectedIssue=LHCBDIRAC-61
   * Web portal first sprint: https://its.cern.ch/jira/secure/RapidBoard.jspa?rapidView=961&view=planning
   * SAM code reviewed: https://savannah.cern.ch/task/index.php?27069
   * New implementation of StorageHistoryAgent https://savannah.cern.ch/task/index.php?28022

| *Date* | *Status* |
| 15/02/2013 | features definition freeze |
| 18/02/2013 | First pre-release on certification. Trunk freezes: only fixes allowed | 
| 06/03/2013 | Last pre-release on certification. Last round of tests launched |
| 11/03/2013 | Release is created and installed in production setup |


---++ Based on DIRAC v6r6:

---+ LHCbDIRAC v7r11

*Features to be included* with link to ITS if available, or with status

Need to be in:
   * ShiftDBAgent adding GEOC to eGroup and sending email to Prod shifter https://savannah.cern.ch/task/index.php?28462
   * Exploiting fully the possibilities given by the production steps: https://savannah.cern.ch/task/index.php?22859
   * Enable popularity reports from WG productions: https://savannah.cern.ch/task/index.php?27709
   * Steps creation - "Option format" field from free text to drop down menu https://savannah.cern.ch/task/?32788
   * Certification procedure first sprint: https://its.cern.ch/jira/secure/RapidBoard.jspa?rapidView=801&selectedIssue=LHCBDIRAC-35

Possibly in:
   * The steps that are 24 months old can be marked as 'obsolete' automatically: https://savannah.cern.ch/task/index.php?27679
   * Put private DQ scripts in LHCbDirac, plus revision: https://savannah.cern.ch/task/?31931

| *Date* | *Status* |
| 23/11/2012 | features definition freeze |
| 27/11/2012 | First pre-release on certification. Trunk freezes: only fixes allowed | 
| 05/12/2012 | Last pre-release on certification. Last round of tests launched |
| 07/01/2013 | Release is created and installed in production setup |

---++ Based on DIRAC v6r4:

---+ LHCbDIRAC v7r10

*Features to be included* with link to Savannah task if available, or with status

   * Script for setting up merging production of RAW (MDF) files
   * Popularity agent should not process users files: https://savannah.cern.ch/task/index.php?32894
   * Put private DQ scripts in LHCbDirac, plus revision: https://savannah.cern.ch/task/index.php?31931 https://its.cern.ch/jira/secure/RapidBoard.jspa?rapidView=603&selectedIssue=LHCBDIRAC-11
   * BKK client calls: https://savannah.cern.ch/task/?35101

| *Date* | *Status* |
| 11/10/2012 | features definition freeze |
| 12/10/2012 | First pre-release on certification. Trunk freezes: only fixes allowed | 
| 16/10/2012 | Last pre-release on certification. Last round of tests launched |
| 17/10/2012 | Release is created and installed in production setup |

---+ LHCbDIRAC v7r9

*Features to be included* with link to Savannah task if available, or with status

   * Using the ByRunFileTypeSizeWithFlush plugin (single merging prod): https://savannah.cern.ch/task/index.php?23192

| *Date* | *Status* |
| 10/09/2012 | features definition freeze |
| 10/09/2012 | First pre-release on certification. Trunk freezes: only fixes allowed | 
| 11/09/2012 | Last pre-release on certification. Last round of tests launched |
| 12/09/2012 | Release is created and installed in production setup |

---+ LHCbDIRAC v7r8

*Features to be included* with link to Savannah task if available, or with status

Need to be in:
   * tasks from 28372 to 28377 - use of Operations helper
   * Application Status containing LFN value: https://savannah.cern.ch/task/index.php?27995
   * Creation of datafile for parametricInputData: https://savannah.cern.ch/support/?128061
   * start run and end run https://savannah.cern.ch/task/?28392
   * CARTESIAN product in the sql command of getFileSummary https://savannah.cern.ch/task/?28400
   * section to be reshuffle https://savannah.cern.ch/task/index.php?28173 <- DELAYED, this change must be done in DIRAC.
   * LCHb TaskManager: adapting to v6r4 https://savannah.cern.ch/task/index.php?28417
   * ShiftDBAgent adding GEOC to eGroup and sending email to Prod shifter https://savannah.cern.ch/task/index.php?28462
   * Reducing the number of BKK calls: https://savannah.cern.ch/task/index.php?30195
   * Correct handle of Logs in failover: https://savannah.cern.ch/task/index.php?30567

| *Date* | *Status* |
| 12/06/2012 | features definition freeze |
| 13/06/2012 | installation on development machine, tests will aim at stable system |
| 15/06/2012 | First pre-release on certification. Trunk freezes: only fixes allowed | 
| 21/06/2012 | Last pre-release on certification. Last round of tests launched |
| 25/06/2012 | Release is created and installed in production setup |

---++ Based on DIRAC v6r3:

---+ LHCbDIRAC v7r7

*Features to be included* with link to Savannah task if available, or with status

Need to be in:
   
   * Runs for BKK queries: https://savannah.cern.ch/task/?27295
   * Bulk queries for getting run informations: https://savannah.cern.ch/task/index.php?27717
   * Augmenting the XML job description with run metadata: https://savannah.cern.ch/task/index.php?27721
   * Template for swimming productions: https://savannah.cern.ch/task/index.php?27296
   * Agent for closing the MC replications: https://savannah.cern.ch/task/?26849
   * Updated dependency: LCGCMT 62b

Possibly in:

   * The steps that are 24 months old can be marked as 'obsolete' automatically: https://savannah.cern.ch/task/index.php?27679


| *Date* | *Status* |
| 13/03/2012 | features definition freeze |
| 16/04/2012 | installation on development machine, tests will aim at stable system |
| 18/04/2012 | First pre-release on certification. Trunk freezes: only fixes allowed | 
| 19/04/2012 | Last pre-release on certification. Last round of tests launched |
| 23/04/2012 | Release is created and installed in production setup |


---+ LHCbDIRAC v7r6

*Features to be included* with link to Savannah task if available, or with status

Need to be in:

LHCbDIRAC (tagged from the v7r5 branch):
   * Fixes for DIRAC-RSS compatibility - on the branch and trunk, tagged: wo_2012040301
   * Templates update for production requests page update (see below) - on the branch and trunk, tagged: wo_2012040301
   * Fix for Failover upload log files - on the branch and trunk, tagged: wo_2012040301
   * run number to be added to ProdConf: https://savannah.cern.ch/task/index.php?27671

LHCbWeb: (tagged from the trunk of LHCbWebDIRAC):
   * Discrete DQFlags in an input data request of a production request: https://savannah.cern.ch/task/index.php?27712

| *Date* | *Status* |
| 03/03/2012 | features definition freeze |
| 04/04/2012 | First pre-release on certification. Branch freezes: only fixes allowed | 
| 04/04/2012 | Last pre-release on certification. Last round of tests launched |
| 05/04/2012 | Release is created and installed in production setup |


---++ Based on DIRAC v6r2:

---+ LHCbDIRAC v7r5

*Features to be included* with link to Savannah task if available, or with status

Need to be in:
   
   * Use of ProdConf: https://savannah.cern.ch/task/?22861
   * Wider possibilities to concatenate steps in a request: https://savannah.cern.ch/task/?27216
   * Two more options in the production step: https://savannah.cern.ch/task/index.php?26793 https://savannah.cern.ch/task/index.php?26794
   * Popularity for user jobs: https://savannah.cern.ch/task/?26841 https://savannah.cern.ch/task/?26836
   * StepAccounting: https://savannah.cern.ch/task/?16839
   * Dual access to RSS and / or CS for Status ( Storage Element ) : https://savannah.cern.ch/task/index.php?25481
   * Banning of SEs: https://savannah.cern.ch/task/?17879
   * ParametricInputData for user jobs: https://savannah.cern.ch/support/?127102

Possibly in:

   * Runs for BKK queries: https://savannah.cern.ch/task/?27295
   * Porting SLS and ConditionDB tests to DIRAC: https://savannah.cern.ch/task/?21239
   * Unify SLS Tables: https://savannah.cern.ch/task/?27009
 
| *Date* | *Status* |
| 16/03/2012 | features definition freeze |
| 19/03/2012 | installation on development machine, tests will aim at stable system |
| 22/03/2012 | First pre-release on certification. Trunk freezes: only fixes allowed | 
| 29/03/2012 | Last pre-release on certification. Last round of tests launched |
| 02/04/2012 | Release is created and installed in production setup |



-- Main.FedericoStagni - 16-Mar-2012
Topic revision: r84 - 2014-08-15 - FedericoStagni
LHCb
Public webs
Welcome Guest
- Cern Search
- TWiki Search
- Google Search
LHCb All webs
Copyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback