TWiki> LCG Web>LCGGridDeployment>LCGSoftware>DataManagementTop>DataManagementWorkPlan (2010-03-11, OliverKeeble) EditAttachPDF
Show Children Hide Children

Data Management
Main
Talks
Components
FTS
LFC
DPM
lcg_util/GFAL
Medical
Last Page Update
OliverKeeble
2010-03-11

This document will soon be removed - please refer to the 2010 Program of Work

LFC and DPM Work Plan

Here is the Data Management Work Plan until the end of the year :

Common to LFC and DPM

Item What ? When ? Who ? Priority Status
1 integration with VOMS/LCAS/LCMAPS   Jean-Philippe/Jiri   DONE
2 Mysql backups   Lana    
3 integrate to build system   Akos   DONE
4 build RPMs on SLC4/ia64   Akos   IN PROGRESS
5 Implement automatic reconnection to database when connection lost   Jean-Philippe   DONE
6 gSOAP 2.7.6 -> 2.7.9   Akos (to provide recipe)    
7 bulk delete function, given a list of GUIDs (ATLAS) End of June Sophie Critical not started
8 limited wildcard support on LFNs for query/delete catalogue operations (ATLAS) End of July Sophie High not started
9 query/delete the catalog content using the combination of list of GUIDs and host field (ATLAS) End of July Sophie Medium not started
10 possibility to modify the "logically deleted" flag (ATLAS) End of July Sophie High not started
11 Python interface for items 7, 8, 9 and 10 End of June/July Robert High not started
12 special thread for special operations (like bulk update of paths oldStorageEndpoint to newStorageEndpoint instead of script run by admin) after items 7, 8, 9, 10, 11 Sophie Medium not started

LFC

Here are listed the requests received about the LFC, together with a timescale estimate.

Item What ? When ? Who ? Priority Status
1 CLI for listreplicax (to get for example the list of files at a given SE)        
3 LFC Mysql catalogue replication       GO SEE PIOTR/NUNO
4 LFC fail over, load balancing       won't do
5 selection of LFC according to the operation (read versus rw)       won't do
6 extension of lcg_util to replace lfc client (lcg_mkdir...)        
7 implement "lfc-ls --replica"     Low  
8 keep case for GUIDs   Jean-Philippe High (LHCb) DONE
11 script to remove the port number in all LFC entries End of July Sophie Medium not started

DPM

Item What ? When ? Who ? Priority Status Needs
1 DPM with disk servers on several network domains (distributed Tier2)   Jean-Philippe   DONE  
2 maximum pin time (possibly with different values depending on VOMS role)          
3 pool hierarchy (+ default pool)          
4 relative path (VO HOME, not chdir)          
5 drain   Jean-Philippe   DONE  
6 (automatic) replication inside a pool          
7 global space reservation with max lifetime (possibly with different values depending on VOMS role)          
8 limit number of streams per disk server (may be useful for some applications like bulk replication, so pool dependent)          
9 port of SRM V2 component to new LBNL wsdl          
10 srmCopy          
11 give the possibility to RFIO access DPM files residing on the client host (needed for AIOD used in gLiteIO)          
12 RFIO client library compatible with CASTOR (old and new)          
13 test of SRM compatibility with FNAL+LBNL          
14 test of multiple instances of SRM daemons with a single backend          
15 test of DPM with multiple disk servers 13/08/2005 Sophie   DONE (tests through SC3) - OK  
16 test of compatibility between dCache client utility srmcp and DPM          
17 implementation of a migrator/recaller to either recall/migrate files automatically between Tier1 and Tier2 or interface to a tape/DVD backend          
18 dpm-gsiftp with last Globus version June 2007 Yaodong + Jean-Philippe   DONE Item 11
19 Perl/Python interface   Jean-Philippe + Sophie   DONE - NEEDS IMPROVEMENTS  
20 DPM dynamic info script   Graeme   DONE Item 19
21 Classic SE -> DPM migration script   Yvan   High TEST THE LAST VERSION
22 Support for Large File Systems in 1.3.8 release Jean-Philippe DONE High (for UK)  
23 Checking file integrity after copy (file size, checksum)       Critical  
24 Python interface to lcg_util/gfal       Critical CHECK WHAT DAVID HAD ALREADY DONE
25 Attributes on replicas       High  
25 Bulk operations       Critical  
26 Find best replica (lcg_util/gfal)       Medium  
27 ACLs propagation between SEs       Medium  
28 Bulk deletion       Critical  
29 Dataset staging       Medium  
30 Quotas          

Last edit: OliverKeeble on 2010-03-11 - 16:44

Number of topics: 1

Maintainer: SophieLemaitre

Edit | Attach | Watch | Print version | History: r18 < r17 < r16 < r15 < r14 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r18 - 2010-03-11 - OliverKeeble
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback