ByteStreamMapping

Introduction

When changing from one TriggerMenu to another old ByteStream datafiles will no longer work properly, as CTPID's, TriggerNames and other attributes are changed. It is therefore desired to have a program that maps the changes from one TriggerMenu to another, thus allowing a transformation of the ByteStream file to mimic data taking with the new TriggerMenu.

The following describes scripts made to map ATLAS ByteStream files from one TriggerMenu to another TriggerMenu. The mapping will change CTPID and TriggerType in both the CTPFragment and the FullEventFragment accordingly.

It should be noted that the transformation is not perfect, as Triggers are omitted or merged from one TriggerMenu to another and as the mapping only changes the CTPID and TriggerType.

It consist of four different scripts:

  • xml_diff.py
  • xml_diff_cosmic1to2.py
  • dumpCTP.py
  • ctpid_printer.py

xml_diff

This script does a CTPID mapping, mapping the CTPID's from the one TriggerMenu to another TriggerMenu. The mapping is done using XML TriggerMenu files, finding the differences between the two menus and generating a python dictionary with the CTPID map in a python file named ctpidmap.py. Furthermore a dictionary for the TriggerType mapping is written in the same file. This has to be modified manually.

There are options to compare both different TriggerMenus from the same version (ie. 1.15 and 1.16) and to compare TriggerMenus from different TriggerMenuVersions (ie Cosmic_2009_v1 to Cosmic_2009_v2).

xml_diff_cosmic1to2

The cosmic1to2 is a version specific for mapping between version 1 and version 2 of the cosmic TriggerMenus. The specific changes are mapping of:

  • RD0_FILLED -> RD0_EMPTY
  • L1_MU0* -> L1_MU0_EMPTY
  • L1_MU6* -> L1_MU6_EMPTY
and for the mbts:
  • L1_MBTS_A -> L1_MBTS_A_EMPTY
  • L1_MBTS_C -> L1_MBTS_C_EMPTY
etc. ...

dumpCTP

The dumpCTP.py changes the actual ByteStream files. As input it takes a ByteStream file and the ctpmap.py generated by xml_diff.py with the relevant TriggerMenus.

The program uses the EventFormat, EventApps and EventStorage (libpyevent_storage) in order to change the CTPIDs and TriggerTypes as stated in the ctpidmap generated with xml_diff.

Specifics of the ByteStreamFile mapping

The mapping of CTPIDs and TriggerTypes will be done in the FullEventFragment and the CTPFragment.

In the FullEventFragment of the binary ByteStream file the CTPIDs are stored in the class 'lvl1_trigger_info()' as a 24bit string(?) where the first 8 words are the CTPIDs before prescale, the next 8 are the CTPIDs after prescale, and the last 8 words are the CTPIDs after veto. The words are a binary representation of a list of decimal numbers - the actual CTPIDs.

  • Example from a 8bit string; [192,0,0,0,0,0,0,0] gives the CTPIDs [6,7] (2**6+2**7 = 192)

After having translated into decimal numbers, the mapping is done, and the CTPIDs is again transformed into the binary representation which is stored in the new datafile. This will be written after all events in the ByteStream file have been processed.

The same procedure is done for the TriggerTypes - taking the dictionary (triggermap) from the ctpidmap file and mapping it accordingly.

In the CTPFragment the mapping is done in the exact same way although CTPIDs are stored in the payload of the fragment, which also contains other information that are not important for the mapping. The CTPFragment is actually the really important place to do the mapping, as it is from here the level 2 triggers gets their information.

Document links

  • The ATLAS Event Format (v4.0): EDMS

ctpid_printer

ctpid_printer is a small program that takes a ByteStream datafile and prints the level 1 identifier, level 1 TriggerType and the CTPIDs of the triggers that fired in the events - both before and after prescale and after veto. If provided with a TriggerMenu file (XML), it will paste the trigger names defined in the TriggerMenu.

Usage

xml_diff

The program is run by typing: python xml_diff.py < arg > < OldTriggerMenuFile > < NewTriggerMenuFile >. The arguments, -a and -s are for comparing TriggerMenus from completely different menus as described above and -sv is for same version.
  • -a: "all" - gives errors and warning as both files and as output in terminal.
  • -s: "simple" - only writes errors and warnings to a file without printing to terminal.
  • -sv: "same version" - for files of same version, as this avoids some false errors that otherwise show up when comparing files from same version.

The created files:

  • ctpidmap.py:
    • A python file with a dictionary of the ctpid mapping and triggernames as comments and a map of triggertypes.
  • errors.dat:
    • Triggers from both files that have NOTHING in common
  • warnings.dat:
    • Triggers were BGRP 0/1 was mapped to BGRP 0 and BPTX 0/1

dumpCTP

The program is run by typing: python dumpCTP.py < Input ByteStream file > ctpidmap < Output ByteStream file > , where ctpidmap is the ctpidmap.py made by xml_diff.py OBS. If you need to change the TriggerType mapping do this manually in the triggermap dictionary in ctpidmap.py before running dumpCTP or in xml_diff.

ctpid_printer

The program is run with: python ctpid_printer.py < Input ByteStream file> ( <menu-file.xml> ) The last argument ( <menu-file.xml> ) can be provided to return TriggerNames as well.

-- AskEmilJensen - 2009-08-31 -- MortenBadensoe - 2009-08-31

Topic attachments
I Attachment History Action Size Date Who Comment
Texttxt ctpid_printer.py.txt r1 manage 3.5 K 2009-08-31 - 10:51 AskEmilJensen ctpid_printer.py
Texttxt dumpCTP.py.txt r1 manage 10.8 K 2009-08-31 - 10:49 AskEmilJensen dumpCTP.py
Texttxt xml_diff.py.txt r1 manage 32.7 K 2009-08-31 - 10:50 AskEmilJensen xml_diff.py
Texttxt xml_diff_cosmic1to2.py.txt r1 manage 45.7 K 2009-08-31 - 10:50 AskEmilJensen xml_diff_cosmic1to2.py
Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r1 - 2009-08-31 - AskEmilJensen
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback