Notes on dCache

Concepts

Storage class

It is an attribute that can be set per directory which defines in which set of pools a file should be stored, or (in the case of a tape system) in which set of tapes.

Cache class

Same thing as the storage class, but has no effect on the tape system.

Spaces

LinkGroups

In short, a LinkGroup is a collection of pools.

Space reservation

DCache however only manages write space, i.e. only space on disk can be reserved and only for write operations. Once files are migrated to tape, and if no copy is required on disk, space used by these files is returned back into space reservation. When files are read back from tape and cached on disk, they are not counted as part of any space. SRM Space reservation can be assigned a non-unique description, then the description cab be used in the future to discover all space reservation with a given description.

In dCache 1.8 each SRM Space Reservation is made against the total available disk space of a particular link group. The total space in dCache that can be reserved is the sum of the available sizes of all Link Groups. If dCache is configured correctly each byte of disk space, that can be reserved, belongs to one and only one Link Group. Therefore it is important to make sure that no pool belongs to more than one pool group, no Pool Group belongs to more than one Link and no Link belongs to more than one LinkGroup.

Explicit and Implicit Space Reservations for Data Storage in dCache

In dCache, if a space reservation is specified, the file will be stored in it (assuming the user has permission to do so in the name space).

If the reservation token is not specified, and implicit space reservation is enabled, then a space reservation will be performed implicitly for each SRM v1.1 and SRM 2.2 srmPrepareToPut or srmCopy in pull mode. If an Access Latency and a Retention Policy are specified, the user defined retention policy and default access latency. If the user has not specified Access Latency or Retention Policy (or if SRM v1.1 is used) , the system will attempt to extract special tags (not surprisingly called “AccessLatency” and “RetentionPolicy”) from PNFS namespace from the directory to which file is being written. If the tags are present, then their values will determine the default Access Latency or Retention Policy that will be used for implicit space reservations. If the tags are not present, then system wide defaults will be used. If no implicit space reservation can be made, the transfer will fail. (Note: some clients also have default values, which are used when not explicitly specified by the user. I this case server side defaults will have no effect. )

Space Manager access control

When SRM Space Reservation request is executed, its parameters, such as reservation size, lifetime, access latency and retention policy as well as user's Virtual Organization (VO) membership information is forwarded to the SRM SpaceManager.

Space Manager uses a special file for listing all the Virtual Organizations (VOs) and all the VO Roles that are permitted to make reservations in the given link group. List of the allowed VOs and VO Roles, together with the total available space and replicaAllowed, outputAllowed, custodialAllowed, onlineAllowed and nearlineAllowed properties of the group is than matched against the information from the user request in order to determine if a given space reservation can be made in particular link group. Once a space reservation is created, no access control is performed, any user can attempt to store the files in this space reservation, provided he or she knows the exact space token.

Configuration

Maximum number of GET TURLs

The parameter srmGetReqMaxReadyRequests defines the maximum number of GET TURLs handed out by the SRM. It is safe to set it to a very large number, like 50000 or 100000. If it is too small, srmPrepareToGet requests can fail.

srmGetReqMaxNumOfRunningBySameOwner

The srmGetReqMaxNumOfRunningBySameOwner parameter affects the job to thread assignment inside the SRM. Setting it to 100 with a thread pool of 1000 basically means that any given user can at most use 100 of those threads. It cannot be used to limit the number of TURLs on a per user basis.

-- AndreaSciaba - 12 Mar 2009