TWiki> CALICE Web>SoftwareMain>TransferToDIRAC (revision 13)EditAttachPDF

CALICE data is being transferred into the ilcdirac catalogue.

we'll update the status on this page.

a little explanation.

  • SE = Storage Element (where the physical file is stored)
  • LFN=Logical File Name (the name in the catalogue, which points to the physical file(s)/replica(s) )
  • PFN=Physical File Name (name of replica on a SE)

1. a "good" case:

  • LFN: /grid/calice/tb-cern/native/cernAhcal2012/Runs_04112012.tgz
  • PFN: srm://dcache-se-desy.desy.de/pnfs/desy.de/calice/tb-cern/native/cernAhcal2012/Runs_04112012.tgz

the PFN is just the LFN (with some SE-dependent prefix).

in this case, we can just create a catalogue entry in dirac.

2. a "bad" case:

  • LFN: /grid/calice/tb-desy-siw-2012/data_skiroc_HVok/sk_beam_sc31_120_3003_19h_noW_3GeV.raw
  • PFN: srm://polgrid4.in2p3.fr/dpm/in2p3.fr/home/calice/generated/2012-04-12/file9ab4a8cf-e837-40b9-b84e-d61d28c1cedc

PFN bears no relation to the LFN: this type of file must physically re-copied using dirac tools.

progress

Plan to migrate the following directories:

/grid/calice/tb-cern

  • 157k files
  • 136k good, at DESY
  • 11k bad filenames

good desy entries are all imported. bad files all copied.

still need to work on in2p3 replicas.

/grid/calice/gain

  • 67k files
  • 44k at DESY
  • 18k at in2p3
  • 4k at *.ac.uk
  • remainder at kek, ciemat

these are all BAD. Are they needed?

/grid/calice/ic_ahcal

155 files, all bad

/calice/ic_ahcal directory does not exist on the DESY SE, and is troublesome to create. Can we copy these somewhere else?

/grid/calice/reco_job

29 files (all bad)

/calice/reco_job directory does not exist on the DESY SE

copied to IN2P3 SE

/grid/calice/SDHCAL

  • 137k files
  • 129k at lyon
  • all BAD

asked Gerald Grenier to transfer these

/grid/calice/software

all "bad"

all copied (to SE IN2P3-USER)

/grid/calice/status

copied to IN2P3 SE

/grid/calice/statusFnal08

copied to IN2P3 SE

/grid/calice/statusFnal08_SCECAL

copied to IN2P3 SE

/grid/calice/statusFnal09

copied to IN2P3 SE

/grid/calice/tb-desy

easy files transferred (good filenames, on the desy SE)

/grid/calice/tb-desy-siw-2012

  • 7013 files
  • 6907 on dcache-se-desy.desy.de
  • 106 on polgrid4.in2p3.fr

all files are "badly" stored.

there are a lot of zero-size files, which are being ignored (not copied into dirac).

all copied (to the SE "IN2P3-USER").

/grid/calice/tb-fnal

  • for files with a single bad replica, it has been re-copied into ilcdirac (DESY-SRM).
  • good cases with a replica at DESY-SRM have been transferred to the dirac catalogue (only the replica at DESY)
  • if there is another replica, it has not yet been added.

in particular, there are many at IN2P3, which are "almost" good

  • -bash-4.1$ gfal-legacy-replicas lfc://grid-lfc.desy.de/grid/calice/tb-fnal/native/dat/Run650423.000.bin
  • srm://ccsrm02.in2p3.fr/pnfs/in2p3.fr/data/calice/tape/tb-fnal/native/dat/Run650423.000.bin
  • srm://dcache-se-desy.desy.de/pnfs/desy.de/calice/tb-fnal/native/dat//Run650423.000.bin

in such cases, the dirac catalogue now points only at the desy replica. I've asked Roman to ask in2p3-cc to move e.g. /calice/*tape*/tb-fnal/native/dat/Run650423.000.bin to /calice/tb-fnal/native/dat/Run650423.000.bin to allow inclusion of also this replica in the dirac catalogue.

/grid/calice/tb-MCProduction

  • all "bad"

Too many files to reimport. Probably most are no longer used. If you want to transfer some of these files, please make an explicit request.

DanielJeansExternal1 - 2020-07-03

Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatext LFC_contents_grid_calice_tb-MCProduction r1 manage 30729.1 K 2020-07-03 - 11:01 DanielJeansExternal contents of tb-MCProduction folder on the LFC
Texttxt LFC_contents_grid_calice_tb-MCProduction_SUMMARY.txt r1 manage 58.2 K 2020-07-03 - 11:01 DanielJeansExternal contents of tb-MCProduction folder on the LFC
Edit | Attach | Watch | Print version | History: r17 | r15 < r14 < r13 < r12 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r13 - 2020-07-08 - DanielJeansExternal
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CALICE All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback