Usage of Tier1-BUFFER storage elements


In order to simplify the operations for processing productions that require access to data that is normally not resident on disk, we propose to use a temporary disk buffer space that needs to be cleaned up when the files are no longer needed.

Tier1-BUFFER storage elements

These SEs are located in the LHCb-Disk SRM space token, with a specific SAPath obtained by appending /lhcb/buffer to the usual SAPath. At CERN it is proposed to locate the CERN-BUFFER storage element on the EOS disk-only service rather than in Castor.


The reconstruction (or reprocessing) jobs would upload the resulting FULL.DST file (replacing the SDST) to their associated Tier1-BUFFER storage, making it available for any stripping production(s).

A standing DM transformation will replicate these datasets to the corresponding Tier1-RDST. This requires a dedicated plugin in the TransformationSystem (ReplicateToLocalSE).

dirac-dms-add-replication --BK /LHCb/Collision12//ReadData/Reco14//FULL.DST --Plugin ReplicateToLocalSE --DestinationSEs Tier1-RDST --Start

A series of stripping productions will be launched on this (re)processing, each corresponding to a specific Processing-Pass (e.g. Stripping10, CaloFemtoDST,...). The corresponding jobs will be directed to sites associated to the Tier1-BUFFER SEs.

Another DM transformation will be launched that will wait until the corresponding files have been processed by all processing passes. This is achieved by a new plugin DeleteReplicasWhenProcessed that accept a series of parameters:

  • SEs where to remove the replicas from (default Tier1BUFFER)
  • List of processing passes to be watched
  • A period at which the plugin should be executed (in order to group requests).
  • A cache life time for caching the list of productions corresponding to the required processing passes (default 24 hours)

dirac-dms-add-replication --BK /LHCb/Collision12//ReadData/Reco14//FULL.DST --Plugin DeleteReplicasWhenProcessed \
                                         --ProcessingPasses Stripping20,CaloFemtoDST --FromSEs Tier1-BUFFER

As soon as a FULL.DST file will be found that has been completely processed, a removeReplica task will be created for removing the Tier1-BUFFER replica. There is no need to launch another removal transformation as long as the list of processing passes doesn't change, as the plugin will look every day for the list of associated productions.

Possible re-use for reprocessing (highly speculative)

With this DeleteReplicasWhenProcessed plugin, one can consider also using it for RAW data reprocessing. There is no need for prompt reconstruction as the files are already staged (although one could also use it in uploading RAW from the pit to CERN-BUFFER). The data flow could be the following:

  • Once a file is staged (without need for pinning, or only for an hour), the stager adds it to a replication transformation using the plugin ReplicateToLocalSE (setting Tier1-BUFFER as destination SEs) that replicates the file from Tier1-RDST to Tier1-BUFFER. The replica in Tier1BUFFER would then preferentially be used by the job when it starts. The cache handling is then much easier as it should only avoid to fill up the disk buffer (this should be taken care of by the plugin as it does now for DSTs). Open issue: how can one set jobs to Waiting only when the file is replicated?

  • A removal transformation waits for the RAW files to be processed for deleting them from the Tier1-BUFFER.

This data flow goes in the direction proposed by the DMS/SM TEG that jobs only access data from disk-only SEs and T1D0 SEs are only used for archival.

An alternative is to use a replication-driven series of transformations: as we should also anyhow develop the possibility to couple FTS transfer requests to the stager (without waiting for FTS-3), a first transformation would replicate (staged) files to Tier1-BUFFER and the actual (re)processing production plugin would only consider files replicated in a Tier1-BUFFER.

-- PhilippeCharpentier - 19-Jun-2012

This topic: LHCb > WebHome > LHCbComputing > UsingBufferSEs
Topic revision: r2 - 2012-06-20 - PhilippeCharpentier
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback