-- MonicaVerducci - 23 Jul 2006

CoolTutorial

Introduction to COOL STANDALONE

Useful webpage to get more information about Atlas Database and COOL deployment in Atlas:

Install and Compile the Package

  • Login on lxplus, configure SCRAM, create working directory

ssh lxplus.cern.ch
tcsh
setenv PATH /afs/cern.ch/sw/lcg/app/spi/scram:${PATH}
mkdir -p WORKDIR/myLCG
rm -rf  WORKDIR/myLCG/COOL_1_3_2
rm -rf  WORKDIR myLCG/myBootStrapFileCoolSRC.COOL_1_3_2
  • 1.Install and build COOL under slc3_ia32_gcc323
setenv SCRAM_ARCH slc3_ia32_gcc323
unsetenv SCRAM_HOME
unsetenv ORACLE_HOME
unsetenv PYTHONPATH
unsetenv ROOTSYS
cd  WORKDIR/myLCG
wget -O myBootStrapFileCoolSRC.COOL_1_3_2 \
  'http://cool.cvs.cern.ch/cgi-bin/cool.cgi/cool/config/scram/BootStrapFileCoolSRC?rev=COOL_1_3_2'
scram project file:myBootStrapFileCoolSRC.COOL_1_3_2
cd COOL_1_3_2/src
scram b

  • 2.Run the UseCases example under slc3_ia32_gcc323 for Oracle and MySQL.

setenv SCRAM_ARCH slc3_ia32_gcc323
cd  WORKDIR/myLCG/COOL_1_3_2/src
eval `scram runtime -csh`
setenv SEAL_CONFIGURATION_FILE   WORKDIR/myLCG/COOL_1_3_2/src/RelationalCool/tests/seal.opts.error
setenv CORAL_AUTH_PATH ${HOME}/private    
exampleCool_UseCases "<oracle URL>"
exampleCool_UseCases "<mysql URL>"
exampleCool_UseCases "<sqlite URL>" | grep -v SQLiteAccess

You may either provide the URL to your favourite servers as a command line argument (between single or double quotes), or change the C++ code and rebuild the sample executable. Run the program with no option to see the syntax:

exampleCool_UseCases

The default setting of the SEAL MessageService in the job options file seal.opts.error is "Error". You may increase the verbosity by creating your own job options file and changing the $SEAL_CONFIGURATION_FILE environment variable. Presently, the COOL plugin exists only for oracle, mysql and sqlite: the URL string, whose syntax is described in detail in the RelationalDatabaseId.h header file, must start by one of "oracle", "mysql" or "sqlite". For Oracle and MySQL, we recommend that you use the "oracle://TNSHOST;schema=SCHEMA;dbname=DBNAME" URL syntax and keep your username and password inside a file ${HOME}/private/authentication.xml (the location of this file is defined in the job options file: it may be modified by using the standard job options and changing the CORAL_AUTH_PATH environment variable, or by creating your own job options file as explained above). The authentication.xml file in the package RelationalCool/tests is an example showing how to declare your favourite username/password pairs for a COOL Oracle or MySQL connection. Alternatively, you may also specify your username and password in the URL string itself, but you should be aware that this option is only kept for backward compatibility and may be unsupported in future releases of COOL. (Note : the new cool_1_3_x versions can be read and modified by Athena, since release 11.5.0., previous releases use cool_1_2_x)

How to run:

1. Go to dir: COOL_1_3_2/src/Examples/UseCases/src
2. Example in UseCases.cpp
3. Modify the code according to the exercise
4. Compile with scram b in the dir COOL_1_3_2/src/Examples/UseCases/src
5. Run with: exampleCool_UseCases "oracle connection"
6. where your oracle connection is:
7. oracle://devdb10;schema=ATLAS_COOL_TUTORIAL;user=ATLAS_COOL_TUTORIAL;dbname=YOURNAME;password=C
ool_tutorial
Then you should type this line of command:
8. exampleCool_UseCases "oracle://devdb10;schema=ATLAS_COOL_TUTORIAL;user=ATLAS_COOL_TUTORIAL;dbname=YOURNAME;password=Cool_tutorial" 

  • EXERCISE 1

  • EXERCISE 2

In my area /afs/cern.ch/user/v/verducci/public there are some useful examples...

Introduction to COOL inside ATHENA

If you are not familiar with ATHENA, look at "Getting Started" in the ATLAS WorkBook before proceeding further. You need an account of the CERN lxplus machines: if you do not already have an account on lxplus, apply for one as soon as possible - look at the ATLAS WorkBook, "Getting Started" for details.

Developer's page: CVS, repository, status of the official code, cmt version, package's tag.

Access to COOL from Athena is done via the Athena IOVDbSvc, which provides an interface between conditions data objects in the Athena transient detector store (TDS) and the conditions database itself. To see COOL writing and reading in Athena, checkout the package AtlasTest/DatabaseTest/IOVDbTestAlg and run the sample top-level joboptions IOVDbTestAlgWriteCool.py and IOVDbTestAlgReadCool.py. (For more information look at the webpage Cool Access from Athena)

Set Up CMT

  • Login to lxplus at CERN and create a working directory called "Tutorial". We will use the release 12.0.0 for this tutorial:

ssh lxplus.cern.ch
cd
mkdir Tutorial
cd Tutorial
mkdir AthenaDB

  • In the directory Tutorial, create a file called "requirements" and setup CMT. My requirements file looks like this where my working area for 12.0.0 is ${HOME}/Tutorial/AtlasDB:

#############################################################
#source /afs/cern.ch/sw/contrib/CMT/v1r18p20051108/mgr/setup.sh

set CMTSITE CERN
set   SITEROOT /afs/cern.ch
set CMTSTRUCTURINGSTYLE "with_version_directory"
#
# Select releases as they are available
#
macro ATLAS_RELEASE   "12.0.0"

#use AtlasSettings v* $(ATLAS_DIST_AREA)

macro ATLAS_DIST_AREA ${SITEROOT}/atlas/software/dist
macro ATLAS_TEST_AREA ${HOME}/

use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)

path_remove  CMTPATH $(HOME)/Tutorial/AtlasDB   
path_prepend CMTPATH $(HOME)/Tutorial/AtlasDB

apply_tag oneTest
#############################################################

Then do the following:

 
source /afs/cern.ch/sw/contrib/CMT/v1r18p20060301/mgr/setup.sh
cmt config
(Note: you can check in the web page the right cmt version for the release!)

Setup for the release 12.0.0

Do the following:
source setup.sh -tag=12.0.0,OneTest
Check that the CMT path is correct by doing this
echo $CMTPATH
You should see on your screen something like:
$(HOME)/Tutorial/AtlasDB:/afs/cern.ch/atlas/software/builds/AtlasOffline/latest (Note:12.0.0 if done -tag=12.0.0,oneTest)
You should see that the path to your working directory, the path to the release directory and the external are set correctly - do not proceed unless this is done correctly.

Run Time Setup

Go to your working area and check out the following package:

cd AthenaDB
cmt co -r IOVDbTestAlg-00-00-46 AtlasTest/DatabaseTest/IOVDbTestAlg/
(Note: with flag -r ; to check out a particular tag otherwise cmt co NAME to check out the head version, use get_tag) Now compile everything as follows:
cd AtlasTest/DatabaseTest/IOVDbTestAlg/IOVDbTestAlg-00-00-46/cmt
cmt config
source setup.sh
cmt broadcast gmake
cd ..
mkdir run
cd run

To test that everything has been done correctly, do:

get_files HelloWorldOptions.py
athena.py -b HelloWorldOptions.py 

You should see on your screen something like:

HelloWorld           INFO execute()
HelloWorld           INFO An INFO message
HelloWorld           WARNING A WARNING message
HelloWorld           ERROR An ERROR message
HelloWorld           FATAL A FATAL error message
AthenaEventLoopMgr   INFO   ===>>>  end of event 9    <<<===
HistorySvc           INFO Service finalised successfully
ChronoStatSvc.f...   INFO  Service finalized succesfully
ToolSvc              INFO Removing all tools created by ToolSvc
ApplicationMgr       INFO Application Manager Finalized successfully
ApplicationMgr       INFO Application Manager Terminated successfully

The Next Time You Login

The next time you login, just do the following:

cd ${HOME}/Tutorial
source setup.sh -tag=oneTest
cd AthenaDB/AtlasTest/DatabaseTest/IOVDbTestAlg/IOVDbTestAlg-00-00-46/cmt
source setup.sh
cd ../run

Examples

Copy the jobOptions (files in the share directory .py) in the directory run, from the directory run, do:

1.   cp ../share/IOVDbTestAlgWriteCool.py  .
2.   cp ../share/IOVDbTestAlgReadCool.py  .
Modify the jobOptions lines IOVDbTestAlgWriteCool.py as follow:

try:
    IOVDbSvc.dbConnection  = "impl=cool;techno=oracle;devdb10:"
    IOVDbSvc.dbConnection  += YOURNAME/ATHENA
    IOVDbSvc.dbConnection  += ":ATLAS_COOL_TUTORIAL:Cool_tutorial"
except:
    IOVDbSvc.dbConnection  = "impl=cool;techno=oracle;devdb10:YOURNAME/ATHENA:ATLAS_COOL_TUTORIAL:Cool_tutorial"

*EXERCISE 3

Implementation of CLOB inside Athena

(Note: features for the cool standalone as insert CLOB in Athena, COOL_IO, and other utilities in the CoolUtilities webpage... )

  • Run a script comand to have a long string CLOB in Athena, see example in my public area: /afs/cern.ch/user/v/verducci/public
1. make_coolstrfile.py
2. pycool_setup.sh

This python utility creates a COOL folder in the specified database ready to hold large string CLOBs as inline payloads, as created and accessed using the Athena MdtCoolStrSvc (in package MuonCondCool). Using CLOBs requires the folders to be created in a special way using the COOL folder.createExtended API call, which is not currently supported in Athena. The command:

 
   source /afs/cern.ch/user/v/verducci/private/pycool_setup.sh
   /afs/cern.ch/user/v/verducci/private/make_coolstrfile.py <database> <folder>
creates a folder in the specified database and sets the folder description string appropriately for an Athena-usable folder.

<database> :  "oracle://devdb10;schema=ATLAS_COOL_TUTORIAL;user=ATLAS_COOL_TUTORIAL;dbname=YOURNAME;password=Cool_tutorial" 
      <folder> : \YOURNAME\FOLDER_NAME

The commands to create a folder for storing CLOBS are:

1. source /afs/cern.ch/user/v/verducci/private/pycool_setup.sh
2. /afs/cern.ch/user/v/verducci/private/make_coolstrfile.py "oracle://devdb10;schema=ATLAS_COOL_TUTORIAL;user=ATLAS_COOL_TUTORIAL;dbname=YOURNAME;password=Cool_tutorial" \YOURNAME\FOLDER_NAME  

Now you can store long string (CLOB) into your database using Athena.

  • EXERCISE 4

Check out the MuonConditions package:

Go to your working area and check out the following package:

cd AthenaDB
cvs co /MuonSpectrometer/MuonConditions/MuonCondUtils/MuonCondCool
(Note:this package is up to now in cvs and not tagged for the release! Do “cvs co” and not “cmt co”)

Now compile everything as follows:

cd MuonSpectrometer/MuonConditions/MuonCondUtils/MuonCondCool/cmt
cmt config
source setup.sh
cmt broadcast gmake
cd ..
mkdir run
cd run
(Note: adding in the requirements file in cmt the line:
use IOVDbTestAlg-00-* AtlasTest/DatabaseTest/IOVDbTestAlg
in this way you compile all packages togheter: IOVDbTestAlg and MuonCondCool doing “cmt broadcast cmt config” you should have all the packages linked)

This package provides MDT Calibration constants to the reconstruction job inside Athena.

  • EXERCISE 5

Some examples are in my public area /afs/cern.ch/user/v/verducci/public

_________________________________________________

Set up Oracle Browser

There are several database browsers freely available, here we suggest some of them: Aquafold, DBVisualizer, Oracle Browser, and Wenaus Oracle Browser.

How to install Aquafold

The installation is well-described in the webpage, following the steps for a linux system. For Linux you will need to enter the following commands in the terminal window. - Change the directory to the location of the downloaded file.

>> cd /home/username/downloads
- Uncompress the package.
>> gzip -d adstudio.tar.gz
- Unarchive the archive file
>> tar -xvf adstudio.tar
- Change the directory to the location of the downloaded file.
>> cd ./Desktop/datastudio
For Linux you may execute the shell script by typing the command in the terminal window ( e.g. >> ./datastudio.sh )

(Note: Java2(TM) Runtime Environment, v. 1.5.0 (JRE) or above installed. You can get the JRE from the Sun web site)

 
Host : oracledb10.cern.ch
Port :10520
SID : d10
Database Name & Password
Query&Analyzer :
select * from NAME  (selection of tables, use "WHERE" to add conditions)
rolleback and commit (to modify tables)

(Note: more information in an ORACLE manual)

  • EXERCISE 6

_______________________________________

• EXERCISE 1

Create a database with a new folder: T0 data. The structure will be: Since – Till -- Channel_id -- Payload where Channel_id is the tube_id, Payload is all the remaining columns. The payload will be composed by: T0 (float), ADC (integer), status (integer), chamber(string) The data are read from an ascii file.

since until Channel_id T0 ADC Status Chamber time time Tube_id (int) float Integer integer string

• EXERCISE 2

Create a database with a new folder: T0CLOB. The structure will be : Since – Till -- Channel_id – Payload where Channel_id is an identifier, Payload is a single long string of formatted data, defined by a header plus the data separeted by a separetor. The payload will be composed by: header, T0 (float), ADC (integer), status (integer) The data are read from an ascii file.

since until Channel_id payload time Time Integer CLOB

• EXERCISE 3

Exercise in Athena. 1.Create a database using IOVDbTestAlgs.cxx. 2.Insert data in different columns (inline payload) as in the exercise 1, and adding in a row a pool root file (reference payload). Modify the code to implement the same structure ad previously done with Cool Standalone.

• EXERCISE 4

Create a new folder to store a CLOB within ATHENA. Run the script: source /afs/cern.ch/user/v/verducci/public/pycool_setup.sh /afs/cern.ch/user/v/verducci/public/make_coolstrfile.py NAME/ATHENA/T0CLOB

• EXERCISE 5

Use the folder created with the previous script and fill it using the package MuonCondUtils, using MdtReadWriteCool.

• EXERCISE 6

Visualize the T0 folder created previously with COOL standalone. Delete some rows and commit the changes.

Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r3 - 2006-07-24 - MonicaVerducci
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback