Introduction
This wiki page presents some results on bulk lfc methods against a LFC which supports encryption.
Bulk methods which have been tested
There are four methods.
- Test 3: int lfc_delfilesbyguid (int nbguids, const char * guids, int force, int * nbstatuses, int * statuses) : allows deleting the replicas and LFNs associated with the given list of guids if the flag force = 1. if force = 0 and there are still replicas for a given guid, nothing is deleted.
- Test 4: int lfc_delfilesbyname (int nbfiles, const char * paths, int force, int *nbstatuses, int *statuses) : allows deleting the replicas and LFNs associated with the given list of LFNs if the flag force = 1. if force = 0 and there are still replicas for a given LFN, nothing is deleted.
- Test 5: int lfc_delfilesbypattern (const char *path, const char *pattern, int force, int *nbstatuses, Cns_filestatus **statuses) : allows deleting the replicas and LFNs if the LFN is like 'path/pattern' and the flag force = 1. if force = 0 and there are still replicas for a given LFN which matches the pattern, nothing is deleted.
- Test 6: int lfc_delreplicas (int nbguids, const char * guids, char * se, int * nbstatuses, int * statuses) allows deleting the replicas only.
Local LFC server and client
Hw and nw properties
- LFC server :lxb0981.cern.ch with MySQL (SL4)
- LFC client : lxb5409.cern.ch (SL4-64)
- ping time 0.34ms
- nb of entries in the DB : 1349134
Test conditions
Tests on different ways to delete files in the LFC. Each test has been repeated three times (same number of entries in the DB).
- Test 1 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs.
- Test 2 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs inside a session.
- Test 3 : lfc_delfilesbyguid to delete both replicas and LFNs (force = 1).
- Test 4 : lfc_delfilesbyname to delete both replicas and LFNs (force = 1).
- Test 5 : lfc_delfilesbypattern to delete both replicas and LFNs (force = 1).
- Test 6 : lfc_delreplicas to delete replicas only.
Test results
Bulk methods which have been tested
There are four methods.
- Test 3: int lfc_delfilesbyguid (int nbguids, const char * guids, int force, int * nbstatuses, int * statuses) : allows deleting the replicas and LFNs associated with the given list of guids if the flag force = 1. if force = 0 and there are still replicas for a given guid, nothing is deleted.
- Test 4: int lfc_delfilesbyname (int nbfiles, const char * paths, int force, int *nbstatuses, int *statuses) : allows deleting the replicas and LFNs associated with the given list of LFNs if the flag force = 1. if force = 0 and there are still replicas for a given LFN, nothing is deleted.
- Test 5: int lfc_delfilesbypattern (const char *path, const char *pattern, int force, int *nbstatuses, Cns_filestatus **statuses) : allows deleting the replicas and LFNs if the LFN is like 'path/pattern' and the flag force = 1. if force = 0 and there are still replicas for a given LFN which matches the pattern, nothing is deleted.
- Test 6: int lfc_delreplicas (int nbguids, const char * guids, char * se, int * nbstatuses, int * statuses) allows deleting the replicas only.
Local LFC server and client
Hw and nw properties
- LFC server :lxb0981.cern.ch with MySQL (SL4)
- LFC client : lxb5409.cern.ch (SL4-64)
- ping time 0.34ms
- nb of entries in the DB : 1349134
Test conditions
Tests on different ways to delete files in the LFC. Each test has been repeated three times (same number of entries in the DB).
- Test 1 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs.
- Test 2 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs inside a session.
- Test 3 : lfc_delfilesbyguid to delete both replicas and LFNs (force = 1).
- Test 4 : lfc_delfilesbyname to delete both replicas and LFNs (force = 1).
- Test 5 : lfc_delfilesbypattern to delete both replicas and LFNs (force = 1).
- Test 6 : lfc_delreplicas to delete replicas only.
Test results
LFC version 1.6.10 without encryption option, LFC server (lxb0769.cern.ch, SL4-32, MySQL). the MySQL database was slow.
Execution time is given in second.
Type of test \ #files to delete |
Test 1 |
Test 2 |
Test 3 |
Test 4 |
Test 5 |
Test 6 |
10 |
6.67 |
0.66 |
0.49 |
0.6 |
26.71 |
0.46 |
100 |
69.41 |
2.57 |
1.92 |
2.57 |
35.68 |
2.13 |
1000 |
668.99 |
34 |
22.16 |
30.69 |
72.9 |
11.7 |
5000 |
3311.56 |
170.6 |
119.91 |
134.18 |
215.24 |
44.28 |
10000 |
6597.8 |
316.55 |
216.31 |
239.78 |
335.27 |
79.22 |
Encryption is off
Execution time is given in second.
Type of test \ #files to delete |
Test 1 |
Test 2 |
Test 3 |
Test 4 |
Test 5 |
Test 6 |
10 |
4.91 |
0.4 |
0.33 |
0.43 |
11.49 |
0.31 |
100 |
49.47 |
1.71 |
0.93 |
1.16 |
12.66 |
0.73 |
1000 |
492.53 |
15.14 |
7.45 |
15.94 |
17.81 |
8.09 |
5000 |
2486.85 |
89.70 |
59.73 |
68.1 |
67.9 |
40.33 |
10000 |
5023.54 |
174.05 |
117.22 |
174.05 |
138.51 |
74.29 |
Encryption is on
Execution time is given in second.
Type of test \ #files to delete |
Test 1 |
Test 2 |
Test 3 |
Test 4 |
Test 5 |
Test 6 |
10 |
4.94 |
0.46 |
0.33 |
0.5 |
11.45 |
0.29 |
100 |
50.01 |
1.67 |
0.99 |
1.19 |
11.8 |
1.09 |
1000 |
505.49 |
16.52 |
13.5 |
16.71 |
16.62 |
8.27 |
5000 |
2473.62 |
93.6 |
59.87 |
66.75 |
68.91 |
34.56 |
10000 |
4956.1 |
180.02 |
116.67 |
126.67 |
126.44 |
59.74 |
--
LanaAbadie - 06 Feb 2008