Dark Data Deletion at FNAL

Introduction

Dark data are old files or datasets, which are physically at FNAL but not registered in Phedex. Fermilab produces dark data list every week. This list is further analyzed and broken into several categories using a darkdata script. The output from the script gives us the overview of potential dark data we can delete at Fermilab. We do further checks and analysis of these dark data lists, then make deletion decisions as documented on this Twiki.

This page is meant to list the dark data we are going to delete, or register in TMDB, or change the status in DBS.

General Check Princples

• Check dataset/file status in DBS
• Check dataset/file if they have higher version and its status in DBS
• Check dataset/file custodial locations

List of dark data campaingns

-- SiLiu - 19-Sep-2012