Site

Question ________________________________Response__________________________________
Site and Endpoints
What is the site name? NDGF-T1
Which endpoint URLs do your archival systems expose? srm://srm.ndgf.org https://dav.ndgf.org and root://ftp1.ndgf.org
How is tape storage selected for a write (choice of endpoint, specification of a spacetoken, namespace prefix). Path or spacetoken
Queue
What limits should clients respect?  
---> Max number of outstanding requests in number of files or data volume In theory unlimited, but not tested above a few million
---> Max submission rate for recalls or queries No limit on rate.
---> Min/Max bulk request size (srmBringOnline or equivalent) in files or data volume As much as fits in an SRM request, 1k - 10k I think it is.
Should clients back off under certain circumstances? Yeah
---> How is this signalled to client? According to SRM standard signalling
---> For which operations? The ones giving error
Is it advantageous to group requests by a particular criterion (e.g. tape family, date)? Not really
---> What criterion? Roughly grouped by time might help a bit, but not much
Prioritisation
Can you handle priority requests? No
---> How is this requested? If this is a strong request from one of our VOs, we would look at implementing it
Protocol support
Are there any unsupported or partially supported operations (e.g. pinning) ? Full support for pinning etc
Timeouts
What timeouts do you recommend? At least a day
Do you have hardcoded or default timeouts? Not user-visible, internal timeouts will be handled by internal retries
Operations and metrics
Can you provide total sum of data stored by VO in the archive to 100TB accuracy? Yes
Can you provide space occupied on tapes by VO (includes deleted data, but not yet reclaimed space) to 100TB accuracy? Yes
How do you allocate free tape space to VOs? We keep track of space in hsminstances in our internal wiki, then set particular hsminstances read-only as tape space runs out
What is the frequency with which you run repack operations to reclaim space on tapes after data deletion? Continuous based on percentage used on a particular tape
Recommendations for clients
Recommendation 1 Request reads in bulk, trickle-feeding requests (a handful every 10 minutes) means we have to implement longer waiting period before we have a reasonable batch of requests to send to retrieval.
---> Information required by users to follow advice  
Recommendation 2 Use SRM, that's the only featureful tape protocol. The others will technically work, but will not have any intelligence.
Buffer Management
Should a client stop submitting recalls if the available buffer space reaches a threshold? No
---> How can a client determine the buffer used and free space?  
---> What is the threshold (high water mark)?  
---> When should the client restart submission (low water mark)?  
If the client does not have to back off on a full buffer, and you support pinning, how is the buffer managed? Restores are queued until there is enough free space in the unlikely event that all space is full
---> Is data moved from buffer to another local disk, either by the HSM or by an external agent? Data is always moved from the primary tape buffer before the user can download it. For this we use the free space on all the VOs disk pools.
Additional questions
Should any other questions appear in subsequent iterations of this survey?  
-- OliverKeeble - 2018-01-30
Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2018-05-07 - MattiasWadenstein
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    HEPTape All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback