Introduction

This wiki page presents some results on bulk lfc methods against a LFC which supports encryption.

Bulk methods which have been tested

There are four methods.

• Test 3: int lfc_delfilesbyguid (int nbguids, const char * guids, int force, int * nbstatuses, int * statuses) : allows deleting the replicas and LFNs associated with the given list of guids if the flag force = 1. if force = 0 and there are still replicas for a given guid, nothing is deleted.
• Test 4: int lfc_delfilesbyname (int nbfiles, const char * paths, int force, int *nbstatuses, int *statuses) : allows deleting the replicas and LFNs associated with the given list of LFNs if the flag force = 1. if force = 0 and there are still replicas for a given LFN, nothing is deleted.
• Test 5: int lfc_delfilesbypattern (const char *path, const char *pattern, int force, int *nbstatuses, Cns_filestatus **statuses) : allows deleting the replicas and LFNs if the LFN is like 'path/pattern' and the flag force = 1. if force = 0 and there are still replicas for a given LFN which matches the pattern, nothing is deleted.
• Test 6: int lfc_delreplicas (int nbguids, const char * guids, char * se, int * nbstatuses, int * statuses) allows deleting the replicas only.

Local LFC server and client

Hw and nw properties

• LFC server :lxb0981.cern.ch with MySQL (SL4)
• LFC client : lxb5409.cern.ch (SL4-64)
• ping time 0.34ms
• nb of entries in the DB : 1349134

Test conditions

Tests on different ways to delete files in the LFC. Each test has been repeated three times (same number of entries in the DB).

• Test 1 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs.
• Test 2 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs inside a session.
• Test 3 : lfc_delfilesbyguid to delete both replicas and LFNs (force = 1).
• Test 4 : lfc_delfilesbyname to delete both replicas and LFNs (force = 1).
• Test 5 : lfc_delfilesbypattern to delete both replicas and LFNs (force = 1).
• Test 6 : lfc_delreplicas to delete replicas only.

Test results

Bulk methods which have been tested

There are four methods.

• Test 3: int lfc_delfilesbyguid (int nbguids, const char * guids, int force, int * nbstatuses, int * statuses) : allows deleting the replicas and LFNs associated with the given list of guids if the flag force = 1. if force = 0 and there are still replicas for a given guid, nothing is deleted.
• Test 4: int lfc_delfilesbyname (int nbfiles, const char * paths, int force, int *nbstatuses, int *statuses) : allows deleting the replicas and LFNs associated with the given list of LFNs if the flag force = 1. if force = 0 and there are still replicas for a given LFN, nothing is deleted.
• Test 5: int lfc_delfilesbypattern (const char *path, const char *pattern, int force, int *nbstatuses, Cns_filestatus **statuses) : allows deleting the replicas and LFNs if the LFN is like 'path/pattern' and the flag force = 1. if force = 0 and there are still replicas for a given LFN which matches the pattern, nothing is deleted.
• Test 6: int lfc_delreplicas (int nbguids, const char * guids, char * se, int * nbstatuses, int * statuses) allows deleting the replicas only.

Local LFC server and client

Hw and nw properties

• LFC server :lxb0981.cern.ch with MySQL (SL4)
• LFC client : lxb5409.cern.ch (SL4-64)
• ping time 0.34ms
• nb of entries in the DB : 1349134

Test conditions

Tests on different ways to delete files in the LFC. Each test has been repeated three times (same number of entries in the DB).

• Test 1 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs.
• Test 2 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs inside a session.
• Test 3 : lfc_delfilesbyguid to delete both replicas and LFNs (force = 1).
• Test 4 : lfc_delfilesbyname to delete both replicas and LFNs (force = 1).
• Test 5 : lfc_delfilesbypattern to delete both replicas and LFNs (force = 1).
• Test 6 : lfc_delreplicas to delete replicas only.

Test results

LFC version 1.6.10 without encryption option, LFC server (lxb0769.cern.ch, SL4-32, MySQL). the MySQL database was slow.

Execution time is given in second.

Type of test \ #files to delete	Test 1	Test 2	Test 3	Test 4	Test 5	Test 6
10	6.67	0.66	0.49	0.6	26.71	0.46
100	69.41	2.57	1.92	2.57	35.68	2.13
1000	668.99	34	22.16	30.69	72.9	11.7
5000	3311.56	170.6	119.91	134.18	215.24	44.28
10000	6597.8	316.55	216.31	239.78	335.27	79.22

Encryption is off

Execution time is given in second.

Type of test \ #files to delete	Test 1	Test 2	Test 3	Test 4	Test 5	Test 6
10	4.91	0.4	0.33	0.43	11.49	0.31
100	49.47	1.71	0.93	1.16	12.66	0.73
1000	492.53	15.14	7.45	15.94	17.81	8.09
5000	2486.85	89.70	59.73	68.1	67.9	40.33
10000	5023.54	174.05	117.22	174.05	138.51	74.29

Encryption is on

Execution time is given in second.

Type of test \ #files to delete	Test 1	Test 2	Test 3	Test 4	Test 5	Test 6
10	4.94	0.46	0.33	0.5	11.45	0.29
100	50.01	1.67	0.99	1.19	11.8	1.09
1000	505.49	16.52	13.5	16.71	16.62	8.27
5000	2473.62	93.6	59.87	66.75	68.91	34.56
10000	4956.1	180.02	116.67	126.67	126.44	59.74

-- LanaAbadie - 06 Feb 2008