WARNING: This web is not used anymore. Please use PDBService.3DFrontierStressTests instead!
 

Frontier Stress Tests

This page describes FroNtier stress tests.

Contact: Luis Ramos

Test Plan

The objectives of this testing effort are:
  • Develop an application and DB schema independent benchmark for Frontier servers
  • Build a tool that identifies performance bottlenecks of a given setup
  • Performance analysis of the complete sw stack
    • CORAL / Frontier plugin
    • Frontier Client
    • Squid
    • Frontier Servlet

The plan is based on the following questions:

  • How fast are the individual components (Database, Application Server, Cache Server and Network)?
  • What impact has different data (content, size, storage type, compression), different complexity database schemas or different caching policies in the performance of the system?
  • How do DB throughput, network bandwidth, payload size, # of clients or server CPU correlate?

The main metrics used in the testing are:

  • Individual and total throughput
  • Server errors
  • CPU consumption/load (clients, frontier server, squid server)
  • Memory usage and disk space needs
  • Network bandwidth usage

The main parameters of testing are:

  • number of client nodes
  • number of test clients
  • payload sizes
  • database structure and content
  • caching policy

Test Infrastructure

Server Setup

The base test infrastructure used is the FroNtier 3D test bed at CERN. These tests use one FroNtier server at frontier3d.cern.ch:8080 and the cooldev database.
The following picture shows the 3D testbed setup used:

3dtestbedtier0_test_setup.JPG

Client Setup

The test clients run in dedicated lxplus nodes with dual Pentium III 1GHz, 500Mb RAM, HD 6Gb and Fast Ethernet (100Mbps). Clients run CORAL_1_5_1 with FrontierClient v2.4.7 (patched to output some of the metrics).
The C++ Persistency/FrontierPlugin test queries the FroNtier server, gathers results and outputs measures. The python controller script starts a number of clients in several client nodes, gathers measures and outputs structured data for plotting.

Further Information

Please find test cases definition and results in the powerpoint version of this page: Status report of FroNtier Stress Tests.


HowTo run the test suite

Using your AFS account login in lxplus and follow the instructions:

  • Checkout from CVS the test suite:

setenv CVSROOT :kserver:@coral.cvs.cern.ch:/cvs/coral
cvs co Tests/Performance_RelationalAccess

The control scripts and configuration file are written in python and are under .../bin.
.../logs/example_report_cli30/index.php is a example report viewable from a web browser. The client test for Persistency is under .../src/CoralPerfClient (needs to be integrated with the Persistency binaries).
If you use Frontier, a patched version of frontier client is supposed to be used.

  • Prepare the Persistency Perf Test in .../tmp

Go to .../tmp and run the script .../bin/setupTest.sh:

cd Tests/Performance_RelationalAccess/tmp
source ../bin/setupTest.sh

  • Configure dblookup.xml
(change only frontier connect string)
   <?xml version="1.0" ?>
   <servicelist>
     <logicalservice name="/db/Frontier">
        <service name="frontier://frontier3d.cern.ch:8080/CoralDev/lramos"
            accessMode="readonly"
            authentication="password" />
     </logicalservice>
   </servicelist>

Note: connect string is frontier://_server_name_.cern.ch:_port_/_servlet_name_/_db_access_schema_
Example: frontier://frontier3d.cern.ch:8080/CoralDev/lramos

  • Configure authentication.xml:
  <connection name="frontier://frontier3d.cern.ch:8080/CoralDev/lramos">
    <parameter name="user" value="lramos" />
    <parameter name="password" value="no_pwd_needed" />
  </connection>
Note: the connect string needs to be the same as in dblookup.xml

  • If using frontier, setup patched frontier client
eval `scram runtime -csh`
hash -r

echo $LD_LIBRARY_PATH
( change entry /afs/cern.ch/sw/lcg/external/frontier_client/2.4.5_cms/slc3_ia32_gcc323/lib to ~/xpto/CORAL_1_5_1/src/FrontierAccess/tests/PerfClient/frontier_client_2_4_7 )

Example: export LD_LIBRARY_PATH=/afs/cern.ch/user/l/lramos/work/CORAL_1_5_1/slc3_ia32_gcc323_dbg/lib:/afs/cern.ch/sw/lcg/app/releases/Persistency/CORAL_1_5_1/slc3_ia32_gcc323_dbg/lib:/usr/lib:/usr/lib64:/usr/lib64:/user/X11R6/lib64:/afs/cern.ch/sw/lcg/external/uuid/1.38/slc3_ia32_gcc323/lib:/afs/cern.ch/sw/lcg/external/Boost/1.33.1/slc3_ia32_gcc323/lib:/afs/cern.ch/sw/lcg/external/mysql/5.0.18/slc3_ia32_gcc323/lib:/afs/cern.ch/sw/lcg/external/oracle/10.2.0.2/slc3_ia32_gcc323/lib:/afs/cern.ch/sw/lcg/external/sqlite/3.3.5/slc3_ia32_gcc323/lib:/afs/cern.ch/sw/lcg/external/Python/2.4.2/slc3_ia32_gcc323/lib:/afs/cern.ch/sw/lcg/external/XercesC/2.7.0/slc3_ia32_gcc323/lib:/afs/cern.ch/user/l/lramos/work/frontier_client_2_4_7_src:/afs/cern.ch/sw/lcg/external/CppUnit/1.10.2_p2/slc3_ia32_gcc323/lib:/afs/cern.ch/sw/lcg/external/expat/1.95.8/slc3_ia32_gcc323/lib:/afs/cern.ch/sw/lcg/external/pcre/4.4/slc3_ia32_gcc323/lib:/afs/cern.ch/sw/lcg/external/../app/releases/SEAL/SEAL_1_8_1/slc3_ia32_gcc323_dbg/lib:/afs/cern.ch/user/l/lramos/work/CORAL_1_5_1/slc3_ia32_gcc323_dbg/lib:/afs/cern.ch/sw/lcg/app/releases/Persistency/CORAL_1_5_1/slc3_ia32_gcc323_dbg/lib:/afs/cern.ch/user/l/lramos/work/CORAL_1_5_1/slc3_ia32_gcc323_dbg/tests/lib:/afs/cern.ch/sw/lcg/app/releases/Persistency/CORAL_1_5_1/slc3_ia32_gcc323_dbg/tests/lib

  • Create test DB
On this example test the DB schema is very simple and the table/fields names are hardcoded in the test code.
After connecting directly to your DB you can:
    • Create the table:
CREATE TABLE T_STRING
( "string" varchar2(4000) )
TABLESPACE "SYSTEM"
LOGGING 
PCTFREE 50
PCTUSED 40
INITRANS 1
MAXTRANS 255
STORAGE (
  INITIAL 16K
  NEXT 97096K
  MINEXTENTS 1
  MAXEXTENTS 505
  FREELISTS 1
  FREELIST GROUPS 1
);

    • Grant select on the table to the frontier user (Frontier3D in frontier3d.cern.ch)
GRANT SELECT ON "LRAMOS"."T_STRING" TO FRONTIER3D;

    • Fill the table
begin
   for i in 1..100000 // for big tests increase this number
   loop
      insert into t_string( "string" ) values ( dbms_random.value( 5, 20 ) );
   end loop;
   commit;
end;

  • Execute unit test
# unitTest_FrontierAccess_CoralPerfClient connStr queryListFilePath reportEveryXSeconds _showFrtMetrics_
unitTest_FrontierAccess_CoralPerfClient /dbdev/Oracle queryList.xml 10 1 | grep "PPP"

note: grep "PPP" -> filter results

Result should be a time regular feed of lines like:
PPP 10000;30000;10600;3;0;#;1.44;1.73;0.00;0.00;4.04;0.00;2.02;4.55;89.39;#;0.82

  • Debug test
export FRONTIER_LOG_LEVEL=debug
export POOL_OUTMSG_LEVEL=D

  • Configure multi-client control script
In ../bin directory you'll find the multi-client control scripts written in python.
Edit ..../etc/perfTestConf.py: change configuration: the test nodes names and change LD_LIBRARY_PATH in the setupCmd variable.

  • Run multi-client control script
python startJobs.py _logDirName_

Example: python startJobs.py test_run_3

note: grep "HHHH" -> filter results

Topic attachments
I Attachment History Action Size Date Who Comment
PowerPointppt lramos-3d-frontierStressTests-StatusReport.ppt r2 r1 manage 309.5 K 2006-10-18 - 17:30 LuisRamos Frontier Stress Tests - Status Report
Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r8 - 2010-06-11 - PeterJones
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    PSSGroup All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback