This page documents procedures to add a new RSE, reconfigure existing RSEs, and to decommission and remove old RSEs

Automatic configuration of RSEs

We have an automated process which can configure RSEs from the storage.json files in SITECONF. This does not run on most sites (yet) by default. To make it run by default add an attribute to the RSE update_from_json=True

This attribute must be added to the test and temp RSEs as well.

rucio-admin rse set-attribute --rse T2_CO_Tier2Name --key update_from_json --value True

If a new RSE must be created, it is enough to create an entry and allow the sync process to fill in the details:

rucio-admin rse add T3_IT_MIB
rucio-admin rse set-attribute --rse T3_IT_MIB  --key update_from_json --value True

This does everything except set the reaper flag to True, which you should do manually. Remember that Temp RSEs are non deterministic so the create code is modified:

rucio-admin  rse add --non-deterministic  T3_IT_MIB_Temp
rucio-admin rse set-attribute --rse T3_IT_MIB_Temp  --key update_from_json --value True

Decommissioning Procedure

Some RSE may stop CMS support and ask for decommissioning of their RSE. Here is a procedure, for e.g. T2_US_Nebraska_Test

1) Clean up of the Production samples

  • Step 1 Get all samples registered at T2_US_Nebraska_Test
  • Step 2 Check whether all samples have at least one custodial replica (2 for RAWs), also make sure that these replicas "OK". If sample unique create rule and transfer data somewhere else.
  • Step 3 Delete all rules one by one rucio delete-rule --all --rse_expression T2_US_Nebraska_Test rule_id (e.g. 15b09ebc47bb47f6973558d0aa3ddbda). You can get all the rules of a certain account by using: rucio list-rules --account sync_t2_us_nebraska where sync_t2_us_nebraska is the account. Then you can use awk or any other linux command line to delete the rules. Ex. $ awk '{system("rucio update-rule " $1 " --lifetime 0")}' rules.txt or $ awk '{system("rucio delete-rule --all --rse_expression T2_US_Nebraska_Test " $1)}' rules.txt where rules.txt is a file with all the rules.

2) Add the mock protocol

  • Step 4 Delete existing protocols. Hostname, prefix and port do not really matter. rucio-admin rse delete-protocol --scheme gsiftp T2_US_Nebraska_Test
  • Step 5 Add new mock protocol for delete: rucio-admin rse add-protocol --hostname mock --scheme mock --prefix / --impl rucio.rse.protocols.mock.Default T2_US_Nebraska_Test --domain-json '{"wan": {"read": 0, "write": 0, "delete": 1, "third_party_copy": 0}}'
  • Step 6 Enable delete and greedy reaper. rucio-admin rse update --rse T2_US_Nebraska_Test --setting availability_delete --value True then rucio-admin rse set-attribute --rse T2_US_Nebraska_Test --key greedyDeletion --value True

How to migrate/update/change the protocols of a RSE

Open a python terminal and do:

from rucio.client import Client
client = Client(account="transfer_ops")
from rucio.client import Client
client = Client(account="transfer_ops")
client.update_protocols("T2_US_Vanderbilt", "gsiftp", {}, old_hostname, old_port)

Into {} add dictionary with “hostname” : “new_hostname”. If you need to see the current setup, you can use:

rucio-admin rse info T2_US_Vanderbilt

Check also the _Test RSE.

Useful updating commands

For more information, take a look here:

rucio-admin rse get-distance T1_ES_PIC_Disk T2_CH_CSCS shows if the link has been setup
rucio-admin rse add-distance --distance 1 --ranking 10 T2_US_Caltech T2_BE_IIHE setups the link between RSEs
rucio-admin account get-limits transfer_ops T2_BE_IIHE shows limit for transfer_ops at given RSE
rucio-admin rse update [-h] [--rse RSE] [--setting PARAM] [--value VALUE] updates any given key and value of RSE
rucio-admin rse set-attribute --rse T1_UK_RAL_Tape_Test --key rule_approvers --value dmielaik RSE rule_approvers update example
from rucio.client import Client
client = Client(account="transfer_ops")
client.update_protocols("T2_US_Vanderbilt_Test", "gsiftp", { "hostname" : "" }, "", 2811)
update protocols

-- DonataMielaikaite - 2020-04-22 -- FernandoGarzon - 2020-07-16

Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r5 - 2021-02-10 - NicholasCharlesSmith1
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback