AskEmilJensen < Sandbox

%TOC%
%STARTINCLUDE%
---+ByteStream mapping

---+Introduction

When changing from one !TriggerMenu to another old ByteStream datafiles will no longer work properly, as CTPID's, !TriggerNames and other attributes are changed. It is therefore desired to have a program that maps the changes from one !TriggerMenu to another, thus allowing a transformation of the ByteStream file to mimic data taking with the new !TriggerMenu.
 
The following describes scripts made to map ATLAS ByteStream files from one !TriggerMenu to another !TriggerMenu. The mapping will change CTPID and !TriggerType in both the CTPFragment and the  FullEventFragment accordingly.  

It should be noted that the transformation is not perfect, as !Triggers are omitted or merged from one !TriggerMenu to another and as the mapping only changes the CTPID and !TriggerType.

It consist of four different scripts: 

   * xml_diff.py
   * xml_diff_comic1to2.py
   * !dumpCTP_0.95.py
   * ctpid_printer_0.2.py

---+xml_diff

This script does a CTPID mapping, mapping the CTPID's from the one !TriggerMenu to another !TriggerMenu. The mapping is done using XML !TriggerMenu files, finding the differences between the two menus and generating a python dictionary with the CTPID map in a python file named =ctpidmap.py=. Furthermore a dictionary for the !TriggerType mapping is written in the same file. This has to be modified manually.

There are options to compare both different !TriggerMenus from the same version (ie. 1.15 and 1.16) and to compare !TriggerMenus from different TriggerMenuVersions (ie Cosmic_2009_v1 to Cosmic_2009_v2). 

---+xml_diff_cosmic1to2

The cosmic1to2 is a version specific for mapping between version 1 and version 2 of the cosmic !TriggerMenus. The specific changes are mapping of:
   * RD0_FILLED -> RD0_EMPTY
   * L1_MU0* ->  L1_MU0_EMPTY
   * L1_MU6* ->  L1_MU6_EMPTY
and for the mbts:
   * L1_MBTS_A -> L1_MBTS_A_EMPTY
   * L1_MBTS_C -> L1_MBTS_C_EMPTY
etc. ...


---+ !dumpCTP

The =dumpCTP.py= changes the actual ByteStream files. As input it takes a ByteStream file and the =ctpmap.py= generated by =xml_diff.py= with the relevant !TriggerMenus.

The program uses the EventFormat, EventApps and EventStorage (libpyevent_storage) in order to change the CTPIDs and !TriggerTypes as stated in the ctpidmap generated with xml_diff.  
 

---++Specifics of the ByteStreamFile mapping

The mapping of CTPIDs and !TriggerTypes will be done in the FullEventFragment and the CTPFragment.

In the !FullEventFragment of the binary ByteStream file the CTPIDs are stored in the class 'lvl1_trigger_info()' as a 24bit string(?) where the first 8 words are the CTPIDs before prescale, the next 8 are the CTPIDs after prescale, and the last 8 words are the CTPIDs after veto. The words are a binary representation of a list of decimal numbers - the actual CTPIDs.

   * Example from a 8bit string; [192,0,0,0,0,0,0,0] gives the CTPIDs [6,7] (2**6+2**7 = 192) 

After having translated into decimal numbers, the mapping is done, and the CTPIDs is again transformed into the binary representation which is stored in the new datafile. This will be written after all events in the ByteStream file have been processed. 

The same procedure is done for the !TriggerTypes - taking the dictionary (triggermap) from the ctpidmap file and mapping it accordingly.


In the !CTPFragment the mapping is done in the exact same way although CTPIDs are stored in the payload of the fragment, which also contains other information that are not important for the mapping. The !CTPFragment is actually the really important place to do the mapping, as it is from here the level 2 triggers gets their information. 

*Document links*

   * The ATLAS Event Format (v4.0): [[https://edms.cern.ch/document/445840/4.0c][EDMS]]

   * The CTPFragment header and the FullEventFragment header: [[http://isscvs.cern.ch/cgi-bin/viewcvs-all.cgi/DAQ/DataFlow/eformat/eformat/?root=atlastdaq][ISSCVS]] 


---+ ctpid_printer
ctpid_printer is a small program that takes a ByteStream datafile and prints the level 1 identifier, level 1 !TriggerType and the CTPIDs of the triggers that fired in the events - both before and after prescale and after veto. If provided with a !TriggerMenu file (XML), it will paste the trigger names defined in the !TriggerMenu.

---+ Usage
---+++ xml_diff
The program is run by typing: =python xml_diff.py < arg > < !OldTriggerMenuFile > < !NewTriggerMenuFile >=.
The arguments, =-a= and =-s= are for comparing !TriggerMenus from completely different menus as described above and =-sv= is for same version.
   * =-a=: "all" - gives errors and warning as both files and as output in terminal.
   * =-s=: "simple" - only writes errors and warnings to a file without printing to terminal.
   * =-sv=: "same version" - for files of same version, as this avoids some false errors that otherwise show up when comparing files from same version.

*The created files:*
   * =ctpidmap.py=:  
      * A python file with a dictionary of the ctpid mapping and triggernames as comments and a map of triggertypes.
   * =errors.dat=:   
      * Triggers from both files that have NOTHING in common
   * =warnings.dat=:
      * Triggers were BGRP 0/1 was mapped to BGRP 0 and BPTX 0/1

---+++ dumpCTP
The program is run by typing: =python dumpCTP.py < Input !ByteStream file > ctpidmap < Output !ByteStream file >= ,
where ctpidmap is the =ctpidmap.py= made by =xml_diff.py=
OBS. If you need to change the !TriggerType mapping do this manually in the triggermap dictionary in =ctpidmap.py= before running !dumpCTP or in xml_diff. 

---+++ ctpid_printer
The program is run with: =python ctpid_printer.py < Input !ByteStream file> ( <menu-file.xml> )=
The last argument ( <menu-file.xml> ) can be provided to return !TriggerNames as well.

-- Main.AskEmilJensen - 2009-08-28
-- Main.MortenBadensoe - 2009-08-28
%STOPINCLUDE%
Topic revision: r4 - 2009-08-31 - AskEmilJensen
Account
- Log In
- Cern Search
- TWiki Search
- Google Search
Sandbox All webs
Edit
Attach
Copyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback