Transfer Scripts

Global CMS checks

global_dataset_replica_check.sh: check dataset replica

• datasets without replica : set their DBS status to DELETED
• global Test and Backfill datasets
• T0 test datasets
• datasets without custodial location

global_production_dataset_check.sh: check datasets still in PRODUCTION status

• production datasets without replica
• Test and Backfill production datasets older than one month
• datasets older than one month, but still in PRODUCTION status

global_GEN_T2_check.sh: check GEN datasets older than 6 months at T2 sites, and submit a deletion request if necessary

1. cron jobs on FNAL transfer machine(cmssrv162)

If you would like to check or edit the cron jobs on cmssrv162, you should log into the machine using the account cmsdatatransfers and your FNAL kerberos password. In general, these cron jobs can be categorized into 2 parts: FNAL regular checks and global CMS checks.

• FNAL regular checks
  FNAL has a huge disk space which we need to manage, there are several categories of data at FNAL (not MC!): custodial datasets to tape, non_custodial datasets to disk.
  We need to make sure that:
  
  • <1> important custodial data is always on disk for fast access => pinning
  • <2> FNAL has a complete copy of all AOD, RECO & RAW of the current and previous data taking period.
  • <3> newly produced samples are transferred to FNAL to complete the copy.
  • <4> everything first into the available disk space.
  • <5> FNAL does not miss any files and there are no orphans