CMS AFS Phase Out

CERN is currently reducing its dependencies on AFS and will eventually phase out this service completely. Please see the IT AFS Phase Out twiki for more details on the phaseout project.

This twiki collects the CMS use cases in AFS and tracks its status to migrate to other alternatives.

If you want to be informed on the progress of AFS phase out in CMS, please register to cms-afsout egroup.

IT Jira Tracker

Specific issues on CMS AFS Phase out actions are already being followed up by CERN IT on JIRA.

Use Cases

acrontab service

About acron replacement due AFS decomission, right now CERN-IT is collecting more information about this service:

  • Is acron an essential service for you / your Experiment?
  • What are your use cases, apart from access to AFS from cron jobs?
  • Do you think that your use cases can be covered by any other IT provided services? If so, which ones?
Please send your feedback to CMS-VOC ( Daniel VSosa ).

CMSSW and Integration Build

  • CMSSW is available in both AFS and CVMFS
    • Some physics groups are accessing CMSSW through webAFS.
  • Integration builds are available on CVMFS
    • It has been observed that integration builds in CVMFS are a bit slow due to some issues handling big files.

Actions Progress
04.10.2016: Shahzad to contact physics groups and understand whether they can stop relying on webAFS and start using other alternatives. Once this is clear, a deadline to stop providing CMSSW through AFS should be defined and announced to the physics groups 29.11.2016 Shazad informs that only one Physics group seems to be affected. It has been suggested to them that they move to WebEOS and update their internal scripts. As of January 2016 no more releases in AFS. They still have some more extra time in any case as they don´t immediately use the latest release.
04.10.2016: Shahzad to follow up with IT on CVMFS issues handling big files 29.11.2016 DONE Non issue after some tests

CMS Web Sites

  • CMS web sites comprise personal and project web sites that will have to migrate to EOS
    • There are come questions on web site support on EOS. Are cgi executables fully supported?

Actions Progress
04.10.2016: Andreas to follow up with IT on EOS web site support and whether it contains all features currently available in AFS RQF0652223
29.11.2016 Andreas to test needed features are available
04.10.2016: Andreas and Shahzad to lead the effort on migrating project and user web sites to EOS 29.11.2016 On hold until previous action completed. Efforts expected early 2017

CERN Siteconf

  • As explained in this twiki CERN T2 siteconfig has some dependencies on AFS that need t be removed.

Actions Progress
04.10.2016: Shahzad and Daniel to implement a solution to stop AFS dependencies on CERN T2 siteconfig 20.10.2016 DONE Thanks to Daniel, siteconf dependency on AFS is removed.

CMS Computing Services

Service Name Dependencies on AFS Responsible Person Comments
CMSWeb YES Bruno  
PhEDEx YES Nicolo

Actions Progress
04.10.2016: Maria and Daniel to complete the list of CMS computing services and organise a meeting with responsible people to identify AFS dependencies 04.11.2016 Daniel to contact service managers to identify possible dependencies

CMS External dependencies

In order to flush still-hidden dependencies from the outside world (as well as potential site misconfigurations), CERN IT proposes to block external access to the CERN.CH AFS cell on January 11th, for a period of 24h (starting at 09:00 CET). Machines on CERN networks will not be affected OTG:0034398

Home Directories

Home directories: a general discussion on this is needed with IT and other experiments.

  • CMS environment is today in AFS. A shared file system is needed that could be accessible by many machines. CEPHFS? ClusterFS? CernBox?
  • If EOS is to be mounted on desktop/VMs, it seems the IP of the machine needs to be whitelisted in the EOS service. How does this work with dynamically allocated IPs?
  • Analysis users build their SW locally and need to share their binaries
  • An EOS public directory per user is needed. However, it seems today in EOS this is not possible since either all or nothing is visible to people. No granularity per directory.

Related to home directories, there is the following linux configuration issue: How will linux manage login scripts sourcing AFS CMS configuration files in AFS?

  • If the machine has CVMFS installed, will use the CMS scripts from there?
  • If not, what is the fallback?

Actions Progress
04.10.2016: Home directories strategy 05.10.2016 Jan contacted by Maria to see whether a common meeting with experiments will be organised to address this. He confirms this is in the plans but there's no date yet
07.11.2016 Strategy on this to be discussed in the IT PoW early november
30.11.2016 Luca is preparing a proposal on how to structure the per-user EOS space, and which parts should be available for syncing


The following issues need further discussion and clarifications with IT:

Actions Progress
29.11.2016: Is CVMFS available from WebEOS? 30.11.2016: Jan suggests to use a container based solution. More documentation on RQF:05871, RQF:0325649, RQF:0661305 and KB:0004358. Andrea will have a look
29.11.2016: Will CERN IT manage the migration of personal web sites from CMS users? Who will make sure that that area “moves correctly” once you move to the home-dir being in EOS space ? Will that (www/) in EOS be handled the same way ? Or will users need to move their AFS/www/ stuff explicitly to some other EOS/www space which is independent of the EOS home dir ?  
29.11.2016: How is CERN IT planning to test first CERN VM image without AFS support, and how do they plan to test also any replacements or alternatives to existing clients or tools that currently rely on AFS, like i.e. cern useradd command? RQF:0678240

Closed issues

Actions Solution
29.11.2016: For EOSUSER/eos/project/c/cmsweb/www/, how subquotas can be handled? Are there any monitoring scripts available? DONE For the start, Andreas will work on some monitoring scripts to control this
04.10.2016: CMS AFS project space: check with IT whether there are guidelines on how to organise this in EOS. DONE CMS EOS is now available under /eos/project/cms

-- MariaALANDESPRADILLO - 2016-10-04

Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r8 - 2017-04-26 - DanielValbuenaSosa
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback