General assumptions

Result set

TOTEM Database stores a large number of very diverse information on a single measurement. These are:
  • Identifier - unique number, which allows to distinguish one measurement from another
  • Single or Vector Value - there are two types of measurements stored in database: the first consist of single value as a value of measurement and a second in which the measurement value is a vector (series of numbers); so if given measurement has single value, it hasn't vector value and vice versa
  • Type of Measurements - describes the measurement, contains: name and description of measurement, unit, single unit and column description which identifies meaning of each value in vector (see above)
  • Interval of Validity - represents a range of events when the kept data is valid (kind of time range); consists of Run number, Start and End events, Start and End times and Type.
  • Timestamp - time of writing data in database

DBEL allows you to specify which kind of information to retrieve. In both version there are appropriate options (console) or fields (web) to provide this functionality.

Time range

Specifying the time range for measurements which would be retrievied from database is optional (if user don't specify time range, all measurements will be retreieved from database). There are three ways of defining the time interval from which the data are to be taken:
  • using timestamps
  • using interval of validity
  • run number
First two are described in section "Intervals of validities vs. Timestamps". The last one allows to retrieve all measurements from given run, which determines long period of time usually associated with some specific settings of LHC.

For timstamps and intervals of validity there is possibility to specify only begin/end restriction or begin and end restriction or nothing.

Intervals of Validity vs. Timestamps

Definitions:

Timestamp
information about single moment, which consists of date and time; format yyyy/mm/dd hh:MM:ss,SSS
where
  • yyyy - year in 4-digit format eg. 2011
  • mm - month in 2-digit format (01-12) eg. 09
  • dd - day in 2-digit format (01-31) eg. 02
  • hh - hours in 2-digit format, 24-hours (00-23) eg. 12
  • MM - minutes in 2-digit format (00-59) eg. 13
  • ss - seconds in 2-digit format (00-59) eg. 14
  • SSS - miliseconds in 3-digit format (000-999) eg. 015

Interval of Validity (IoV)
it is a structure for storing information about time range (interval), where kept data is valid. It represents a set of data (eg. measurements) related in some way. It consist of ID, begin and end timetsamps. The ID is an artificial number, created by DB's sequence, and is used to identify a single IoV. Begin and end timestamps determine timeframe of the interval (boudaries of section). IoV has also information about run number.

Run number
identifies a huge amount of data, which are related somehow (eg. by the some optics configuration during data acquisition). Generally single run number contains a lot of IoVs.

Motivation:

DBEL provides two ways of specifying time ranges for the data, which user want to retrieve:

  • using timestamps
  • using interval of validities
But what is the difference between them? Why there are two ways?

Solution:

In TOTEM database for every general measurement there are two types of information:

  • timestamp
  • interval of validity
The first determines the exact time when this measurement was obtained (from DAQ).
The second is used for defining when this measurement (for how long) is valid. Very often the begin timestamp is equal to the single timestamp described previous, but it is not a rule. So when user is interested in exact data acquistion time more then data from whole set related in some way the appropiate method for retrieving data is using Timestamp. But when user prefer the data for given period (as said previuos this measurements may be interrelated) more than exact timeframe the better method is using intervals of validity. But how to specify IoV?
The essential problem is that to specify IoV (which is kind of period of time) we should use its ID, which is unknown by the user. Specifying IoV by begin and end timestamps is also difficult, because user could not have such information or there is high probability of mistake (two timestamps should be given to the nearest miliseconds). In order to make DBEL simplier in usage, user can identify IoV by single timestamp. How? In TOTEM database IoV don't overlap. So they are in kind of sequnce. Using this characteristic user can supply only one timestamp from given interval of validity (which is contained between begin and end timestamps from IoV), and demanded IoV will be founded by DBEL.

Below there is a picture showing the difference between using timestamps and intervals of validity.

iov vs timestamp.png
Intervals of validity vs. Timestamps

Description:

Assume that in our database there are two highlighted IoVs (black sections). Between them there are also other IoVs, as well before and after them.

Green and red arrows represent two points in time (two timetsamps).

The dashed section represents the set of data (measurements) which will be retrieved.

On the left side in the picture we have 4 symbolic names: t1, t2, i1, i2.

They represents timestamps (t) and intervals of validity (i). The number 1 means begin and 2 means end.

So this picture show what we get (from which period of time) when we specify various combinations of timestamps or intervals of validities.

For specyfing timestamp and intervals of validities in console version of DBEL we use options: -t1, -t2, -i1, -i2. In web version there are appropriate fields (with sliding calendars).

Remember: when you specifiy single date for intervals of validities, it means that this interval will be selected which has an intersection with given datetime.

Number of results

Some queries can return huge data sets. In order to limit number of retrieved measurements user can specify parameter called number of results. This option limits the result set of measurements to given number. If the number of measurement in result set is lower than given nothing will happen. Specifying the number of results cuts the result set to the given number and returns the given number of first subsequent measurements of result set.

But what if we want different measurements from result set? There is another option - first result number. This will skip the given number of measurements from the measurements retrieved from database (omission given number measurements in the result set, counting from beginning of result set). If number of results isn't specified, all measurements which corresponds to the query will be retrieved. But if the number of measurements in result set will be too big (size exceeds 1000MB), data won't be extracted from database, and appropiate message will be display.

Description of the query

Very often users work simultaneously with many XML files (created by DBEL). They can store these files on local disk for a long time. But there is a possibility that after some time, users could not remember what kind of measurements they store in these files (which measurement, from what period of time). In order to minimize this risk DBEL allows to put description of the query into output XML file (description is in self-commenting XML format). This description is put at the begining of the file, so user will be able to recall specification of the query with minimal effort.

-- JakubSzymanek - 02-Sep-2011

Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng iov_vs_timestamp.png r1 manage 1110.7 K 2011-09-02 - 15:16 UnknownUser Intervals of Validity vs. Timestamps
Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2011-09-06 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    TOTEM All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback