Difference: RawDataFormat (1 vs. 5)

Revision 52019-07-08 - CristovaoDaCruzESilva

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Line: 128 to 128
 

Eudaq

Added:
>
>
(n.b. The current implementation/description assumes that the EUDAQ is only being used with the mimosa detectors. It is possible to use with some other detectors, but would require changes to the code at several levels)
 The Eudaq readout plugin receives UDP packets from EUDAQ and stores them in the raw event data stream as a fragment. The first word of the UDP packets from EUDAQ must be the value 0x0d15ea5e and the last word must be the value 0xd15ea5e5. These two words are stripped and are not placed in the fragment inside the raw data stream.
Line: 139 to 141
 The EUDAQ has a Model ID of 0x00000800. Each UDP packet has its own header, with the first word being the IP from where the UDP packet was sent and the second word the total size of the UDP packet + header. There may be more than one UDP packet in a EUDAQ fragment.
Changed:
<
<
Typically one would expect 2 UDP packets, one for each readout frame of the detectors, however it is possible to configure EUDAQ to only send one frame or there might be network timeouts, meaning that the number of packets may vary between 0 and 2.
>
>
Typically one would expect 2 UDP packets, one for each readout frame of the mimosa detectors, however it is possible to configure EUDAQ to only send one frame or there might be network timeouts, meaning that the number of packets may vary between 0 and 2.
 Please note however that timeouts during data taking most likely are a symptom of some other issue in the setup.

Position Explanation Example

Revision 42019-07-02 - CristovaoDaCruzESilva

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Line: 128 to 128
 

Eudaq

Changed:
<
<

Micromegas

>
>
The Eudaq readout plugin receives UDP packets from EUDAQ and stores them in the raw event data stream as a fragment. The first word of the UDP packets from EUDAQ must be the value 0x0d15ea5e and the last word must be the value 0xd15ea5e5. These two words are stripped and are not placed in the fragment inside the raw data stream. This plugin is agnostic of the data that EUDAQ is actually sending, even though at the moment only the Mimosa detectors are used in this way. This means that in the RCDRawEvent class the EUDAQ fragment is automatically assumed to be for a mimosa detector and is decoded as such. If new detectors are implemented from the EUDAQ side, this will most likely require adding a status word to identify which detector (most likely within the UDP packet itself) and then modifying the RCDRawEvent class to correctly marshal the data.

The EUDAQ has a Model ID of 0x00000800. Each UDP packet has its own header, with the first word being the IP from where the UDP packet was sent and the second word the total size of the UDP packet + header. There may be more than one UDP packet in a EUDAQ fragment. Typically one would expect 2 UDP packets, one for each readout frame of the detectors, however it is possible to configure EUDAQ to only send one frame or there might be network timeouts, meaning that the number of packets may vary between 0 and 2. Please note however that timeouts during data taking most likely are a symptom of some other issue in the setup.

Position Explanation Example
+0 The source identifier 0x0051000a
+1 Model ID, always the same value 0x00000800
+2 Size of the fragment 0x00000020
+3 Start of first UDP packet. The IP that sent the packet 0x83a985d2
+4 The length of the UDP packet +2 0x0000000d
... The UDP packet itself 0x10000003
... The UDP packet itself 0x00000001
... The UDP packet itself 0x55515555
+M Start of second UDP packet. The IP that sent the packet 0x83a985d2
+M+1 The length of the UDP packet +2 0x0000000f
... The UDP packet itself 0x11000003
... The UDP packet itself 0x00000001
... The UDP packet itself 0x55515555
N-1 Fragment Footer 0xc0badebb

MMFE8 (Micromegas)

 

Event End Block

Revision 32019-06-28 - CristovaoDaCruzESilva

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Line: 107 to 107
 
GEO[4:0] 0 1 0 CRATE[7:0] 0 0 CNT[5:0] NA

The V792 data words are bitfields, see below.

Changed:
<
<
The most relevant quantities are bits 26-24 which identify the word as being a data word
>
>
The most relevant quantities are bits 26-24 which identify the word as being a data word, bits 20-16 which indicate what channel the data refers to and bits 11-0 which give the ADC reading. Bits 13 and 12 might also be useful for indicating conditions that may have happened on the channel.
 Note that for the 16 channel V792 (the V792 N) the CHANNEL field is one bit shorter.

Version 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
Line: 116 to 117
 

The V792 trailer word is a bitfield, see below.

Changed:
<
<
The most relevant quantities are bits 26-24 which identify the word as being a trailer
>
>
The most relevant quantities are bits 26-24 which identify the word as being a trailer and bits 23-0 which give an event counter, typically it should be the same as the event number.
 
31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
Added:
>
>
GEO[4:0] 1 0 0 EVENT COUNTER[23:0]
 

V1290 (TDC)

Revision 22019-06-27 - CristovaoDaCruzESilva

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Added:
>
>

Raw Data Format

In 2019, there was a restructuring and consolidation of the code, meaning that some of the format of the events has changed, hopefully for the better. However, the old format does not completely represent the data format prior to 2019, this because there were some modification which were not documented. Still, the changes are minor such as the IDs for the modules which are slightly different.

After 2019

The raw data format can be seen as a sequence of blocks. In this section, the words are referred to with an offset, the offset is with respect to the start of the block. The offset is in units of words, not bytes. The raw file starts with a first block describing the run/file and is not documented here.

General description and block layout

The data from sequential events is separated by the so-called separator blocks. Each separator block is typically followed by an event start block. The event start block is followed by the readout module blocks. The number and position of these blocks depends on what is configured to be read out; see the OKS database for details. The readout module blocks share some layout in order to facilitate the reading of the data and of unknown readout types. This is important and if you are implementing a new module, make sure to follow it. After the readout module blocks there is the event end block, marking the end of the data.

Position Block
0 Separator Block
1 Event Start Block
... Readout Module Block
... Readout Module Block
... Readout Module Block
(N-1) Event End Block

Separator Block

Position Explanation Example
+0 Separator Marker (aka Header ID), always the same value 0x1234cccc
+1 Number of words in separator (aka size), includes the first word. In this version, should always be the same value 0x00000004
+2 Number of blocks of data so far (in a simple setup, should match the event number +1) 0x000000a3
+3 Size of all the event blocks in bytes (why?????) before the next separator, does not include the separator block 0x0000014c

Event Start Block

In the event start block, there seems to be a sub-block. This sub block originates from the RCDApp and contains some run-related information. This sub-block, just like the blocks from the readout modules has as the first word a source identifier. The 2 high bytes of the source identifier are the sub-detector ID. The 2 low bytes of the source identifier are the Module ID. Both IDs can be configured in the OKS database, the Module ID (currently 0x0054) under the RCDApp and the sub-detector ID (currently 0x0051) from the General_TileSegment.data.xml file, since for BL4S a simplified database is used with a single generic sub-detector.

Position Explanation Example
+0 Event Start Marker (aka Event Start ID), always the same value 0xee1234ee
+1 Number of words in block (aka size), includes the first word. In this version, should always be the same value 0x00000009
+2 The format version of the data 0x03010000
+3 Start of the RCDApp sub-block. The source identifier 0x00510054
+4 Run number 0x5cfa80b6
+5 Level 1 ID (currently used as the event number) 0x00000001
+6 Bunch crossing ID (in the BL4S setup, this seems to always match the Level 1 ID) 0x00000001
+7 Level 1 trigger type (seems to be unused in BL4S) 0x00000000
+8 Detector event type (seems to be unused in BL4S) 0x00000000

Readout Module Blocks

Every readout module block should contain a header and a footer. The data from the module should be placed after the header and before the footer. The specific format of the data depends from module to module.

The first word of the header should always be the source identifier. The 2 high bytes of the source identifier are the sub-detector ID. The 2 low bytes of the source identifier are the Module ID. Both IDs can be configured in the OKS database, the Module ID under the corresponding channel configuration and the sub-detector ID (currently 0x0051) from the General_TileSegment.data.xml file, since for BL4S a simplified database is used with a single generic sub-detector.

The second word of the header is an ID to identify which type of module, called Model ID. These IDs are in the source code of the plugins implementing the modules, see the rcd_**** under RCDTDAQ.

The third word of the header should be the size of the block in words. This includes the 3 header words and 1 footer word.

The footer contains a single word which should always have the value 0xc0badebb.

For the specific modules, see below:

V792 (QDC)

The QDC traditionally used in BL4S has 32 channels. There does exist a V792 with only 16 channels, but it has not been used in the scope of BL4S. If it is ever used, some adaptations to the code might be required.

The V792 has a Model ID of 0x00000300. Typically the V792 reads out all the channels, but it seems that it can on occasion miss a few, so the data packet itself will always be 32 words or smaller. The 32 channels (1 word of data for each) plus the 3 words from the header, the one word from the footer and the 1 header and 1 trailer from the QDC data explain why the fragment size is typically 0x00000026, which is 32+6=38.

Position Explanation Example
+0 The source identifier 0x00510001
+1 Model ID, always the same value 0x00000300
+2 Size, typically has the value 0x00000026 0x00000026
+3 Start of V792 data. The V792 defines a header word 0xfa012000
... Typically 32 words with the V792 data format 0xf8004037
N-2 End of V792 data. The V792 defines a trailer word 0xfc000014
N-1 Fragment Footer 0xc0badebb

The V792 header word is a bitfield, see below. The most relevant quantities are bits 26-24 which identify the word as being a header and bits 13-8 (CNT) the number of channel (i.e. words) that have been read.

31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
GEO[4:0] 0 1 0 CRATE[7:0] 0 0 CNT[5:0] NA

The V792 data words are bitfields, see below. The most relevant quantities are bits 26-24 which identify the word as being a data word Note that for the 16 channel V792 (the V792 N) the CHANNEL field is one bit shorter.

Version 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
V792 GEO[4:0] 0 0 0 NA CHANNEL[4:0] NA UN OV ADC[11:0]
V792 N GEO[4:0] 0 0 0 NA CHANNEL[3:0] NA UN OV ADC[11:0]

The V792 trailer word is a bitfield, see below. The most relevant quantities are bits 26-24 which identify the word as being a trailer

31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0

V1290 (TDC)

V560 (Scaler)

Eudaq

Micromegas

Event End Block

For the event end block, the positions are counted from the end... for some reason. Depending of the value of the last word, there are two possible layouts (although I have only seen the first one with BL4S). I have still not figured out where that status words come from nor how to configure how many exist.

Layout 1

Position Explanation Example
... Status Word 0x00000000
... Status Word 0x00000000
-2 Number of Status Words 0x00000002
-1 Size of all the readout module blocks in words (n.b. does not include the event start and event end blocks) 0x00000045
+0 Status Position(0: Status -> Data, 1: Data -> Status) 0x00000001

Layout 2

Position Explanation Example
-(N-1) Size of all the readout module blocks in words (n.b. does not include the event start and event end blocks) 0x00000045
... Status Word 0x00000000
... Status Word 0x00000000
-1 Number of Status Words 0x00000002
+0 Status Position(0: Status -> Data, 1: Data -> Status) 0x00000000

Old format

 
Word Explanation
1234cccc Seperator Marker
00000004 Number of words in seperator

Revision 12016-07-14 - OskarWyszynski

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="WebHome"
Word Explanation
1234cccc Seperator Marker
00000004 Number of words in seperator
003d01a0 Number of following block of data within this file sequence
00000198 Size of block before next seperator (there are 49 words, this marker shows 198 = 49*4, which means it is number of bytes)
   
ee1234ee Event Start header marker
00000009 Number of words in header
03010000 Format version no (same with real atlas events)
00510054 Source identifier (0051: sub det, 0054: Module ID)
5418751b Run No (Repeated in whole file)
003d019f Level 1 ID
003d019f Bunch Crossing ID
00000000 Level 1 Trig Type
00000000 Detector Event Type
   
00510002 Channel ID (v792)
00000792 Model ID
00000022 Number of words in next block (34: 32 data + 1 header + 1 EOB)
fa012000 QDC Data words(Details in DataChannelV792.cpp)
f8004036 ...
f810404f  
f8014054  
f811406f  
f8024056  
f812405c  
f803406a  
f8134075  
f8044068  
f814405c  
f8054052  
f815405d  
f80640ab  
f8164077  
f8074077  
f8174068  
f808405e  
f818408a  
f8094072  
f8194072  
f80a407f  
f81a407c  
f80b406b  
f81b406f  
f80c406c  
f81c406c  
f80d406e  
f81d4083  
f80e4079  
f81e4070  
f80f4075  
f81f4079  
fc3d05ee QDC Trailer
   
00510003 Channel ID(1st v1290)
00001290 Model ID
47a033ff TDC Data Words(Details in DataChannelV1290.cpp)
0819f119 ...
00001333  
00601309  
0080173d  
00c011ac  
00201c12  
00401758  
008028e8  
00e0203b  
000052af  
00a023fa  
1819f00c  
0919f119  
010017c9  
01600c99  
01201c9e  
0140207a  
1919f006  
8000029f Global TDC trailer
   
00510006 Channel ID(2nd v1290)
00001290 Model ID
47a033ff TDC Data Words(Details in DataChannelV1290.cpp)
0819f215 ...
0080069f  
00c00873  
00a005ed  
1819f005  
0919f215  
01000691  
1919f003  
8000015f Global TDC trailer
   
00510004 Channel ID(v560)
00000560 Model ID
00000010 Scaler Data Words(Details in DataChannelV560.cpp)
003d01a0  
00638db6  
0221ec1e  
02178a1a  
02302813  
01636ba4  
060a8b03  
00638dac  
00000000  
00000000  
00293f71  
024cd812  
00000000  
00000000  
00000000  
00000000  
   
00000000 Status Word 1
00000000 Status Word 1
00000000 Status Word 1
00000000 Status Word 1
00000004 Number of Status Words
00000056 Data Words in the event (86 = 36(QDC)+21(TDC1)+11(TDC2)+18(Scaler)
00000001 Status Position(0: Status -> Data, 1: Data -> Status)
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback