Introduction

This wiki page presents some results on bulk lfc methods against a LFC which supports encryption.

Bulk methods which have been tested

There are four methods.
  • Test 3: int lfc_delfilesbyguid (int nbguids, const char * guids, int force, int * nbstatuses, int * statuses) : allows deleting the replicas and LFNs associated with the given list of guids if the flag force = 1. if force = 0 and there are still replicas for a given guid, nothing is deleted.
  • Test 4: int lfc_delfilesbyname (int nbfiles, const char * paths, int force, int *nbstatuses, int *statuses) : allows deleting the replicas and LFNs associated with the given list of LFNs if the flag force = 1. if force = 0 and there are still replicas for a given LFN, nothing is deleted.
  • Test 5: int lfc_delfilesbypattern (const char *path, const char *pattern, int force, int *nbstatuses, Cns_filestatus **statuses) : allows deleting the replicas and LFNs if the LFN is like 'path/pattern' and the flag force = 1. if force = 0 and there are still replicas for a given LFN which matches the pattern, nothing is deleted.
  • Test 6: int lfc_delreplicas (int nbguids, const char * guids, char * se, int * nbstatuses, int * statuses) allows deleting the replicas only.

Local LFC server and client

Hw and nw properties

  • LFC server :lxb0981.cern.ch with MySQL (SL4)
  • LFC client : lxb5409.cern.ch (SL4-64)
  • ping time 0.34ms
  • nb of entries in the DB : 1349134

Test conditions

Tests on different ways to delete files in the LFC. Each test has been repeated three times (same number of entries in the DB).
  • Test 1 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs.
  • Test 2 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs inside a session.
  • Test 3 : lfc_delfilesbyguid to delete both replicas and LFNs (force = 1).
  • Test 4 : lfc_delfilesbyname to delete both replicas and LFNs (force = 1).
  • Test 5 : lfc_delfilesbypattern to delete both replicas and LFNs (force = 1).
  • Test 6 : lfc_delreplicas to delete replicas only.

Test results

Bulk methods which have been tested

There are four methods.
  • Test 3: int lfc_delfilesbyguid (int nbguids, const char * guids, int force, int * nbstatuses, int * statuses) : allows deleting the replicas and LFNs associated with the given list of guids if the flag force = 1. if force = 0 and there are still replicas for a given guid, nothing is deleted.
  • Test 4: int lfc_delfilesbyname (int nbfiles, const char * paths, int force, int *nbstatuses, int *statuses) : allows deleting the replicas and LFNs associated with the given list of LFNs if the flag force = 1. if force = 0 and there are still replicas for a given LFN, nothing is deleted.
  • Test 5: int lfc_delfilesbypattern (const char *path, const char *pattern, int force, int *nbstatuses, Cns_filestatus **statuses) : allows deleting the replicas and LFNs if the LFN is like 'path/pattern' and the flag force = 1. if force = 0 and there are still replicas for a given LFN which matches the pattern, nothing is deleted.
  • Test 6: int lfc_delreplicas (int nbguids, const char * guids, char * se, int * nbstatuses, int * statuses) allows deleting the replicas only.

Local LFC server and client

Hw and nw properties

  • LFC server :lxb0981.cern.ch with MySQL (SL4)
  • LFC client : lxb5409.cern.ch (SL4-64)
  • ping time 0.34ms
  • nb of entries in the DB : 1349134

Test conditions

Tests on different ways to delete files in the LFC. Each test has been repeated three times (same number of entries in the DB).
  • Test 1 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs.
  • Test 2 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs inside a session.
  • Test 3 : lfc_delfilesbyguid to delete both replicas and LFNs (force = 1).
  • Test 4 : lfc_delfilesbyname to delete both replicas and LFNs (force = 1).
  • Test 5 : lfc_delfilesbypattern to delete both replicas and LFNs (force = 1).
  • Test 6 : lfc_delreplicas to delete replicas only.

Test results

LFC version 1.6.10 without encryption option, LFC server (lxb0769.cern.ch, SL4-32, MySQL). the MySQL database was slow.

Execution time is given in second.
Type of test \ #files to delete Test 1 Test 2 Test 3 Test 4 Test 5 Test 6
10 6.67 0.66 0.49 0.6 26.71 0.46
100 69.41 2.57 1.92 2.57 35.68 2.13
1000 668.99 34 22.16 30.69 72.9 11.7
5000 3311.56 170.6 119.91 134.18 215.24 44.28
10000 6597.8 316.55 216.31 239.78 335.27 79.22

Encryption is off

Execution time is given in second.
Type of test \ #files to delete Test 1 Test 2 Test 3 Test 4 Test 5 Test 6
10 4.91 0.4 0.33 0.43 11.49 0.31
100 49.47 1.71 0.93 1.16 12.66 0.73
1000 492.53 15.14 7.45 15.94 17.81 8.09
5000 2486.85 89.70 59.73 68.1 67.9 40.33
10000 5023.54 174.05 117.22 174.05 138.51 74.29

Encryption is on

Execution time is given in second.
Type of test \ #files to delete Test 1 Test 2 Test 3 Test 4 Test 5 Test 6
10 4.94 0.46 0.33 0.5 11.45 0.29
100 50.01 1.67 0.99 1.19 11.8 1.09
1000 505.49 16.52 13.5 16.71 16.62 8.27
5000 2473.62 93.6 59.87 66.75 68.91 34.56
10000 4956.1 180.02 116.67 126.67 126.44 59.74

-- LanaAbadie - 06 Feb 2008

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2008-02-12 - LanaAbadie
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback