Site Support Team - Documentation
Removing a CMS Site

Morgue: Make sure the site is IN the morgue (down for production) and no jobs or data is being sent to it anymore.
Compute Elements of Site: Decommission the dedicated CMS CEs
- Remove queues from factories --> ticket to glideinfactory (osg-gfactory-support@physicsNOSPAMPLEASE.ucsd.edu or cms-ggus-glideinfactory@cernNOSPAMPLEASE.ch)
Storage Elements of site Contact transfer team to relocate data and disable transfers to the site, i.e.:
- SE Data: Ensure that all data on your SE is safely replicated to a custodial location
- VO CMS: Remove support for the VO CMS from SEs
  - Done in BDII (Site Admin)
- CMS Data: Delete all CMS data from site.
  - Through Phedex (Data Manager of the Site)
  - Deletion request of all replicas/subscriptions at the site.
  - Contact Users & Groups that have files on Site (/store/user & /store/group) (Data Manager of the Site or Site Admin)
  - If the deletion is stuck because the site is not running agents anymore:
    - Transfer Team has to do a local invalidation of all datasets on the site (or global if it was the last copy) before proceeding with the removal in the usual way.
- Phedex agents: Stop Phedex agents (Site Admin)
- Phedex endpoints:
  - Remove Phedex endpoints and revoke PhEDEx DB roles (GGUS ticket to Transfer team)
  - Remove all links to/from the site in Prod/Debug/Dev (can be done by the transfer team)
  - Rename site to XT2_PL_Cracow (should be done by Natalia )
  - Remove the DB roles for T2PLCRACOW (should be done by Natalia )
  - If an SE is renamed or changed: Site has to change that information in SiteDB, BDII and their storage.xml and site-local-config.xml. After that send an email to the factories and also to Natalia to update the TMDB manually with the new SE name.
Check/Update BDII Information
- check if site still advertises CMS support,
```
ldapsearch -x -h lcg-bdii.cern.ch -p 2170 -b mds-vo-name=local,o=grid -LLL '(|(GlueSAAccessControlBaseRule=cms)(GlueSAAccessControlBaseRule=VO:cms)(GlueSAAccessControlBaseRule=VOMS:/cms/*)(GlueCEAccessControlBaseRule=VO:cms)(GlueCEAccessControlBaseRule=VOMS:/cms/*))' | grep Glue | grep UniqueID
```
  If yes, then
- lookup email address of site admin in SiteDB
- send email to site admin asking her/him to remove support for VO CMS from site level BDII (advise that site executive should remove site completely from BDII if CMS was last/only VO supported)
- CMS does no longer use BDII, so this is for consistency only
Squids: Stop monitoring squids
- http://wlcg-squid-monitor.cern.ch/snmpstats/indexcms.html has the list of sites with monitoring squids.
- If they site is located on this list, email cms-frontier-support@cernNOSPAMPLEASE.ch to remove it
SITECONF: Remove the Site from SITECONF in GIT
- https://twiki.cern.ch/twiki/bin/view/CMSPublic/SiteConfInGitlab#Instructions_to_delete_a_project
SiteDB: Remove from SiteDB
- ask Stephan or email other person with permissions to remove sites from SiteDB
- https://cmsweb.cern.ch/sitedb/prod/sites
- If the site still appears in the "List of CMS site names and the SAM/BDII name used by WLCG" it is because the site is still in SiteDB
  - https://cmsweb.cern.ch/sitedb/reports/showXMLReport?reportid=cms_to_sam.ini
  - v1 of SiteDB still thinks that the site is there
  - SiteDB responsible (Bruno Moreira Coimbra - bruno.moreira.coimbra@cernNOSPAMPLEASE.ch ) has to manually remove the site.
Dashboard & SSB: contact Dashboard team to remove site and to identify metrics that still feed info into SSB for the site
- Ask the metric responsible to stop providing data for the site
  - Phedex(transfer team) ( http://vocms039.cern.ch/aaa-fedinfo/federations.json )
  - HammerCloud (Andrea Sciaba)
  - #good links in the site readiness view (Andrea Sciaba)
  - Production (Workflow team, cms-comp-ops-workflow-team@cernNOSPAMPLEASE.ch)
  - Site readiness (Site Support Team), the site could be excluded from /data/cmssst/SiteReadiness/input/excluded-sites.input on the manager machine.
  - Claimed/Unclaimed Pilots (SI Team, cms-submission-infrastructure@cernNOSPAMPLEASE.ch)
  - Federation (Transfer Team, cms-comp-ops-transfer-team@cernNOSPAMPLEASE.ch) (https://cmsweb.cern.ch/phedex/datasvc/json/prod/subscriptions?node=T2_CH_CSCS_HPC&create_since=1)
  - Pledges, Production Cores and Real Cores (Ignore at the moment)
- Delete dashboard data of the site for the last two months (if site was in drain that long) or wait for site to flush out
  - to delete data, right-click on cell in site readiness and select "Remove Cell from Database"
  - site should disappear from Site-Readiness after about two month (script will remove site after all feeding metrics provides no data for about a month)
  - site should disappear from Site-Readiness View depending on the Tier it belongs to. Tier 2 takes longest, three months.
  - another way to delete dashboard data is to run script:
```
 python deleteSiteInfo.py -c 154 -s T2_RU_RRC_KI -f "2015-01-01" -t "2015-02-02" 
```
    where -c is the column from which you wish to delete data -s is the site and -f and -t are from and to dates in which to delete the data. follow the steps and it'll create a file named dataToDelete.txt. Then create an unencrypted copy of your key at .globus with
```
openssl rsa -in  ~/.globus/userkey.pem -out  ~/.globus/unencr_key.pem
```
    and finally
```
cat datatoDelete.txt | xargs -n 1 curl -k -H "Accept : application/json" -X GET --cert ~/.globus/usercert.pem --key ~/.globus/unencr_key.pem
```
    That last step actually deletes the data, using the unencrypted cert for authentication with the dashboard web service.
GGUS: close out tickets and prevent new tickets
- Close out all tickets for the site.
- If appropriate, remove the site from the CMS Site selector and grid site selectors by making a ticket and asking the GGUS team to delete the entry from the internal database