New Small Wheel Database

Software required and installation

The ongoing db/server code is stored at : https://gitlab.cern.ch/formica/NswSac

The ongoing GUI code is stored at : https://gitlab.cern.ch/jcosta/NswSac_GUI

Working implementations pre-require (at least) :

git java jdk maven oracle jbdc driver python pyQt

Testing SQL database with h2 :

Start by launching the server as usual with h2 : ~nsw-web> mvn -Dspring.profiles.active=jetty,h2,prodDB jetty:run

Then kill the application, you should have a .db file under /tmp/ for the nsw db.

You can then go to the m2 directory where the jar file for the h2 database is (in my case it's under /Users/jcosta/.m2/repository/com/h2database/h2/1.4.18) : ~thatPlace > java -jar h2-1.4.182.jar

A Browser will appear and you need to fill the following lines : URL JDBC: jdbc:h2:/tmp/nswspringdb

And you can start playing with you local h2 database for SQL tests !

SQL code (tests)

/* CREATE TABLE WORKFLOWSTEPSJOAO(STEP_ID VARCHAR PRIMARY KEY, STEPNAME VARCHAR, NEXTSTEP VARCHAR); */ ALTER TABLE WORKFLOWSTEPSLOGBOOK ADD INSERTIONTIME TIMESTAMP CREATE TABLE WORKFLOWSTEPSLOGBOOK(LOG_ID INT PRIMARY KEY, C_ID VARCHAR, STEP_ID VARCHAR);

SELECT * FROM WORKFLOWSTEPSLOGBOOK

SELECT * FROM NSW_COMPONENT

/* Basic components */ INSERT INTO WORKFLOWSTEPSJOAO VALUES('RECEPTION','Reception of component','CONTROL') INSERT INTO WORKFLOWSTEPSJOAO VALUES('CONTROL','component controlled','ACCEPTED') INSERT INTO WORKFLOWSTEPSJOAO VALUES('ACCEPTED','component controlled',NULL) /* Module */ INSERT INTO WORKFLOWSTEPSJOAO VALUES('ASSEMBLY','Assembly stage','VALIDATION') INSERT INTO WORKFLOWSTEPSJOAO VALUES('VALIDATION','Validation stage','PREPTOCERN') INSERT INTO WORKFLOWSTEPSJOAO VALUES('PREPTOCERN','Prepared to transport to CERN',NULL) -- panels INSERT INTO WORKFLOWSTEPSJOAO VALUES('PREPARATION','Preparation of panel stage','CONSTRUCTION') INSERT INTO WORKFLOWSTEPSJOAO VALUES('CONSTRUCTION','construction of panel stage','FINALISATION') INSERT INTO WORKFLOWSTEPSJOAO VALUES('FINALISATION','construction of panel stage',NULL)

ALTER TABLE WORKFLOWSTEPSLOGBOOK ADD FOREIGN KEY (STEP_ID) REFERENCES WORKFLOWSTEPSJOAO (STEP_ID);

/* operations on the table components where the original is what spring has */ ALTER TABLE NSW_COMPONENT ADD T_ID VARCHAR ALTER TABLE NSW_COMPONENT ADD FOREIGN KEY (T_ID) REFERENCES NSW_COMPONENT_TYPE (T_ID);

ONGOING

SELECT * FROM NSW_WORKFLOW_STEPS; SELECT * FROM NSW_COMPONENT_TYPE ; SELECT * FROM NSW_LOCATION ; SELECT * FROM NSW_COMPONENT ; SELECT * FROM WORKFLOWSTEPSLOGBOOK ; SELECT * FROM NSW_COMP_TO_TYPE ; SELECT * FROM NSW_MEASUREMENT ; SELECT * FROM NSW_MEASUREMENT_DATA ; SELECT * FROM NSW_COMP_LOC ;

CREATE TABLE WORKFLOWSTEPSLOGBOOK(LOG_ID INT PRIMARY KEY, C_ID VARCHAR, STEP_ID VARCHAR); ALTER TABLE WORKFLOWSTEPSLOGBOOK ADD FOREIGN KEY (STEP_ID) REFERENCES NSW_WORKFLOW_STEPS (STEP_ID); ALTER TABLE WORKFLOWSTEPSLOGBOOK ADD DESCRIPTION VARCHAR;

ALTER TABLE NSW_WORKFLOW_STEPS ADD NEXTSTEP_ID VARCHAR;

ALTER TABLE NSW_COMPONENT ADD T_ID VARCHAR ; ALTER TABLE NSW_COMPONENT ADD FOREIGN KEY (T_ID) REFERENCES NSW_COMPONENT_TYPE (T_ID);

ALTER TABLE NSW_MEASUREMENT ADD DESCRIPTION VARCHAR; ALTER TABLE NSW_MEASUREMENT ADD COMMENTS VARCHAR;

ALTER TABLE NSW_MEASUREMENT_TYPE ADD DATA_CONSTRAIN VARCHAR; ALTER TABLE NSW_MEASUREMENT_TYPE ADD T_ID VARCHAR; ALTER TABLE NSW_MEASUREMENT_TYPE ADD FOREIGN KEY (T_ID) REFERENCES NSW_COMPONENT_TYPE ;

/* Continue */ /* Basic components */ INSERT INTO NSW_WORKFLOW_STEPS VALUES('RECEPTION',CURRENT_TIMESTAMP(),'Reception of component','CONTROL') ; INSERT INTO NSW_WORKFLOW_STEPS VALUES('CONTROL',CURRENT_TIMESTAMP(),'component controlled','ACCEPTED') ; INSERT INTO NSW_WORKFLOW_STEPS VALUES('ACCEPTED',CURRENT_TIMESTAMP(),'component controlled',NULL) ; /* Module */ INSERT INTO NSW_WORKFLOW_STEPS VALUES('ASSEMBLY',CURRENT_TIMESTAMP(),'Assembly stage','VALIDATION') ; INSERT INTO NSW_WORKFLOW_STEPS VALUES('VALIDATION',CURRENT_TIMESTAMP(),'Validation stage','PREPTOCERN'); INSERT INTO NSW_WORKFLOW_STEPS VALUES('PREPTOCERN',CURRENT_TIMESTAMP(),'Prepared to transport to CERN',NULL); /* panels */ INSERT INTO NSW_WORKFLOW_STEPS VALUES('PREPARATION',CURRENT_TIMESTAMP(),'Preparation of panel stage','CONSTRUCTION') ; INSERT INTO NSW_WORKFLOW_STEPS VALUES('CONSTRUCTION',CURRENT_TIMESTAMP(),'construction of panel stage','FINALISATION'); INSERT INTO NSW_WORKFLOW_STEPS VALUES('FINALISATION',CURRENT_TIMESTAMP(),'construction of panel stage',NULL) ;

/* -- Adding some basic components */ INSERT INTO NSW_COMPONENT VALUES ('BLS1',CURRENT_TIMESTAMP(), CURRENT_TIMESTAMP(), 'a RO Board Stereo','BS'); INSERT INTO NSW_COMPONENT VALUES ('BLS2',CURRENT_TIMESTAMP(), CURRENT_TIMESTAMP(), 'a RO Board Stereo','BS'); INSERT INTO NSW_COMPONENT VALUES ('MAPI1', CURRENT_TIMESTAMP(), CURRENT_TIMESTAMP(), 'a insert pin', 'MAPI'); INSERT INTO NSW_COMPONENT VALUES ('FRRO1', CURRENT_TIMESTAMP(), CURRENT_TIMESTAMP(), 'a raw pin', 'FRRO'); INSERT INTO NSW_COMPONENT VALUES ('RL1S', CURRENT_TIMESTAMP(), CURRENT_TIMESTAMP(), 'a RO Panel Stereo', 'RS');

/* Doing some component-location associations */ INSERT INTO NSW_COMP_LOC VALUES(1,CURRENT_TIMESTAMP(), FALSE, 'IRONMAN', 'BLS1', 'STORAGE'); INSERT INTO NSW_COMP_LOC VALUES(2,CURRENT_TIMESTAMP(), FALSE, 'IRONMAN', 'BLS2', 'STORAGE'); INSERT INTO NSW_COMP_LOC VALUES(3,CURRENT_TIMESTAMP(), FALSE, 'IRONMAN', 'MAPI1', 'STORAGE'); INSERT INTO NSW_COMP_LOC VALUES(4,CURRENT_TIMESTAMP(), FALSE, 'SILVERSURFER', 'FRRO1', 'CERN'); /* Doing some component-type associations */ INSERT INTO NSW_COMP_TO_TYPE VALUES(1,CURRENT_TIMESTAMP, 'BLS1', 'BS'); INSERT INTO NSW_COMP_TO_TYPE VALUES(2,CURRENT_TIMESTAMP, 'BLS2', 'BS'); INSERT INTO NSW_COMP_TO_TYPE VALUES(3,CURRENT_TIMESTAMP, 'MAPI1', 'MAPI'); INSERT INTO NSW_COMP_TO_TYPE VALUES(4,CURRENT_TIMESTAMP, 'FRRO1', 'FRRO');

/* Some function to see how it works */ select AVG(PK_ID) from NSW_COMP_TO_TYPE ;

/* Doing some workflow */ INSERT INTO WORKFLOWSTEPSLOGBOOK VALUES (1,'BLS1','RECEPTION','Waiting to be received. Someone told it's foreseen tomorrow') ; INSERT INTO WORKFLOWSTEPSLOGBOOK VALUES (2,'BLS1',select NEXTSTEP_ID from NSW_WORKFLOW_STEPS where STEP_ID='RECEPTION','package received. Waiting for control validation') ; INSERT INTO WORKFLOWSTEPSLOGBOOK VALUES (3,'BLS2','RECEPTION','Waiting to be received. Someone told it's foreseen tomorrow') ; INSERT INTO WORKFLOWSTEPSLOGBOOK VALUES (4,'BLS2',select NEXTSTEP_ID from NSW_WORKFLOW_STEPS where STEP_ID='RECEPTION','package received. Waiting for control validation') ; INSERT INTO WORKFLOWSTEPSLOGBOOK VALUES (5,'BLS2','CONTROL','Being controlled') ;

/* Some queries */ SELECT UTF8TOSTRING(DATA) FROM NSW_MEASUREMENT_DATA WHERE HASH = SELECT MEASUREMENT_HASH FROM NSW_MEASUREMENT WHERE PK_ID=2;

SELECT STEP_ID FROM WORKFLOWSTEPSLOGBOOK WHERE LOG_ID = (SELECT MAX(LOG_ID) FROM WORKFLOWSTEPSLOGBOOK WHERE C_ID = 'BLS1') ;

/* evolve workflow automatically */ INSERT INTO WORKFLOWSTEPSLOGBOOK VALUES (6,'BLS1',select NEXTSTEP_ID from NSW_WORKFLOW_STEPS where STEP_ID=(SELECT STEP_ID FROM WORKFLOWSTEPSLOGBOOK WHERE LOG_ID = (SELECT MAX(LOG_ID) FROM WORKFLOWSTEPSLOGBOOK WHERE C_ID = 'BLS1')),'package received. Waiting for control validation') ;

/* virtual table, basically read-only */ CREATE VIEW view_name AS SELECT STEP_ID,C_ID,LOG_ID FROM WORKFLOWSTEPSLOGBOOK ;

/* find current workflow step for each component (incomplete) */ SELECT A.C_ID,MAX(A.LOG_ID) FROM WORKFLOWSTEPSLOGBOOK A GROUP BY A.C_ID; /* above I cannot have the name of the step .... and in h2 inner joins dont work ... actually join operations are not working well */

ATHENA generic commands

Useful scripts

checkFile.py myFile.pool.root => Gives index of contents of pool.root file; (B) = branches, (T) = trees

** that check differences in ESD(AOD?) files diffPoolFiles.py --> this is comparing container sizes

acmd.py diff-root file1 file2 --> this is comparing branch by branch (much more detail)

Job Transforms commands

Reco_trf.py

preExec='rec.doNameAuditor=True' ==> see what are the clients of tools (?)

preExec='rec.doFloatingPointException=True' ==> look for fpe

preExec='rec.OutputLevel=DEBUG' ==> dump outputs with DEBUG mode

postExec='topSequence.TimeOut=0' ==> define timeout time for algorithms in athena

CMT/SVN

cmt commands :

cmt show clients --> shows clients of a given package (takes a long time)

cmt show versions Trigger/TrigMonitoring/TrigMuonMonitoring ==> Shows which tag version in current release

PROC RELATED

Standard PROC commands

* To setup a nightly release : asetup 17.1.X.Y-VAL,rel_1 (this is for instance if monday nightly for 17.1.X.Y-VAL)

Typical example of how to check tag differences and RTT job differences between nightlies :

asetup 17.1.X.Y-VAL,rel_5

get-tag-diff.py --ref=17.1.X.Y-VAL,rel_1 --chk=17.1.X.Y-VAL,rel_5

compareTCTs.py --nightly=17.1.X.Y-VAL --rRef=rel_1 --rVal=rel_5 >! log_compareTCT_rel1_rel5

Standard twikis/egroups/hn to follow

Daily work links

Typical ways to check RTT running:

bjobs -u rtt | less

https://atlasrtt.cern.ch/admin/batchjobs/

Useful scripts

du -sh dir // human-readable size of directory named dir

T0/eos/castor file finding

retrieving files from eos (example) : If files are in atlascerngroupdisk,

STAGE_HOST=castoratlast3 STAGE_SVCCLASS=atlascerngroupdisk

rfcp root://eosatlas//eos/atlas/atlascerngroupdisk/proj-sit/tct/rec_input/00183021/physics_MinBias/data11_7TeV.00183021.physics_MinBias.merge.RAW._lb0550._SFO-ALL._0001.1 .

(general link) https://twiki.cern.ch/twiki/bin/view/Atlas/ATLASStorageAtCERN#Writing_and_reading_files_on_EOS

needs cleaning

In a folder A there are several packages, how to compile them all in 1 turn ?

in the test area, setupWorkArea.py cd WorkArea/cmt and cmt bro make that wil compile all

--updating AtlasDBRelease-- The only thing you need to do is to update DB Release version number in the requirements

--inputFilePeeker--

athena.py -i

from AthenaCommonFlags import athenaCommonFlags athenaCommonFlags.FilesInput=['myFilePath'] from InputFilePeeker import InputFileSummary inputFileSummary['desiredtypedata_itemsDic'] => for example 'metadata_itemsDic, or eventdata_itemsDic

SUSY

NAF

Accessing desy's computers via NX

> nxdesy ( accesses to atlas-wgs02 opening a connexion)
> open nx client (select desy)

If there's connexion timeout, connect to atlas-wgs02 directly and do : > ps -x | grep nx
and clear out any existing processes. Then reconnect again to nxdesy and nx client.

From atlas-wgs to naf

open a shell and execute "source toNAF.sh"

STOP analysis

https://savannah.cern.ch/task/?group=atlas-susy-prod
Edit | Attach | Watch | Print version | History: r21 < r20 < r19 < r18 < r17 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r21 - 2016-05-12 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback