This activity has been started with the ATLAS Tier3 workshop (25-26 of January 2010).

Tier3 contacts

Tier 3 Technical Coordinator DougBenjamin

Tier 3 Coordinator AndrejFilipcic

Tier3 working groups

As an outcome of the preparation meetings, 6 working groups have been started (February 2010) and finished their work by mid summer 2010. Some groups will continue others will be merged into new working group as the need arises.

twiki page describing 6 Atlas Tier 3 working groups from Winter/Spring 2010

Warning, important The 6 working groups information was moved to this page AtlasTier3WorkingGroupsSpring2010

Beyond the 6 working groups

Setting up a Tier 3 at your institution

Your Tier 3 can be setup in many ways. The exact choice will depend on what you would like to do with it. Some Tier 3's are colocated with existing Tier 2 facilities. Other Tier 3 sites are stand alone. These sites can either be Tier 2 like with full complement of grid middle-ware, all of the Atlas services (Production queues and Atlas DQ2 services). Such a site requires a significant effort (> 0.5 FTE) to maintain a quality site. Another approach for stand alone Tier 3 sites are ones that have far less of grid infrastructure and require much less effort to maintain ( < 0.25 FTE). (Tier 3g)

Help Atlas Tier 3g Guide - The description of a Tier 3g including instructions for setting up a Tier 3g site can be found on these twiki pages

Tier 3 management

Tier-3 computing is locally controlled, non pledged computing resources. The primary focus of Tier-3 computing is defined by local needs. Some Tier-3 sites may focus on analysis while others provide computing resources to the ATLAS collaboration through the central production system.

Tier-3 project is one component of ATLAS Distributed Computing (ADC) activities. Operational and technical aspects are discussed and presented within ADC. All non-technical issues are treated and approved by the ATLAS International Computing Board (ICB).

This project is managed by Tier-3 coordinator and Tier-3 Technical coordinator. The roles and responsibilities are listed bellow.

Tier-3 Coordinator mandate

Tier-3 Coordinator is the primary contact between ADC and ICB on activities related to the Tier-3 project. The Tier-3 coordinator and technical coordinator will develop the ATLAS Tier-3 policy. This Tier-3 policy will be approved by the ICB with concurrence by ATLAS Computing management and ADC.

As new sites request to join the Tier-3 project, they should contact Tier-3 coordinator and present their site details. Tier-3 coordinator will verify that the site is in compliance with established ATLAS policy. The site information will be forward onto the ICB in a manner appropriate for the site.

The Tier-3 Coordinator is responsible for actions related to any changes in site categorization (for example promoting Tier-3 site to ATLAS production site).

The Tier-3 Coordinator and the Tier-3 technical coordinator will establish a Tier-3 communication forum and define communication channels.

Tier-3 Technical Coordinator mandate

The Tier-3 Technical Coordinator leads the Tier-3 project within ADC. The technical coordinator ensures that the Tier-3 project has a minimal impact on the central operations.

The technical coordinator assists the sites with technical issues associated with site operation.

The technical coordinator will participate in new technologies evaluation that could be used in the Tier-3 community.

The technical coordinator and Tier 3 coordinator will present software development requests and new features requirements to the ADC.

The technical coordinator and Tier 3 coordinator work together on operational issues related to Tier-3 sites.

The roles and responsibilities will evolve as the Tier-3 project evolves.

Technical Requirements

T3 Architecture

  • T3 Co-located: T2/T1 shared resources
  • T3gs: with grid services Full setup (CE,SE), T2 alike Partial (SE only, CE only)
  • T3g: grid access (client)

T3 Categories

  1. High priority production: T2-like jobs
  2. Distributed analysis
  3. Low priority production: priority<200, simulation only
  4. User directed analysis: no central brokering

T3 Site requirements

Category Cores Storage WAN Support
1 High priority production 50 20TB 1Gb/s 1 FTE
2 ATLAS-wide analysis 50 50TB 1Gb/s 1 FTE
3 Low priority production 25 10TB 100Mb/s 0.5 FTE
3 User directed analysis 25 10TB 100Mb/s 0.25 FTE

The numbers represent a minimum recommendation by ADC. If the site does not meet the requirements, it's status must be negotiated with ICB and cloud support.

The category 1-3 site must run the grid services.

Checkpoint of April the 21st

How can I see who is already involved? How can I make my participation visible?

  • We have inventories. Let's decide to have a stable home for these (stable twiki location?)
    • And a mechanism to keep them up-to-dated
  • As soon you can run HammerCloud, you will be visible also via this

How can I help? how can I explain it to my colleagues, funding, etc...

  • I think we are missing a small description of what a Tier3 is, prototype status etc... (coarse descritotion)
  • Missing an entry point to get involved in the prototype work
  • Write a CHEP abstract!

Hardware choices

  • The starting point is what you have already got
    • Guidelines (prepared for the US sites) are of general interest
      • DELL offer
    • Publish your hardware (see Inventories)

Technology choices (Data Management)

  • The starting point is what you have already got (within reason)
    • Main technologies are Xroot and Lustre (or other distributed file system like GPFS)
    • If you start from scratch please consider only Xroot or Lustre

Technology choices (All the rest)

  • Follow the installation instructions
    • Report bugs or unclear stuff
    • Contribute to the documentation
    • Coherence and uniformity! Do not go your own way without telling and without reason!!!
  • Virtualisation best-practices


  • See the "Technology choices" part

How does my Tier3 get data?

  • dq2-get works out of the box (test and small Tier3 only)
  • dq2-get on steroids exist (beta version)
    • Contact your local Tier1 to get a proper FTS channel set up. Start to ask for it immediately since it can take some time
    • Discourage subscription at this point

Which data should I download?

  • Your Tier3 hosts the data you want
  • For testing purposes, some datasets to exercises HammerCloud should be downloaded (and HammerCloud will say your site does not work)
  • The download of other datasets depend on your local-users needs

Where should I put these data?

  • The original idea is to put all files in the "same" place
  • We need a working decision today to go ahead
  • Avoid to drift (too much) away from simplicity and uniformity

Is there a repository for the Tier3 distribution?

  • How can we leverage OSG/VDT or other sources

How do I get ATLAS software?

  • Coherent with the "grid" installation (tools reuse + integrated with others)
  • Shared-file system or CVMFS?
    • In the later case FUSE on the WN is needed

How do I get conditions?

  • CVMFS for files
    • Fuse on the WN needed?


  • Inventory of examples (ATLAS specific) has to be created
  • What to do with sites without a batch system (very small sites)
  • What to do with sites with a batch system (from very small to very large)

How can I demonstrate my Tier3 is used/useful

  • Accounting
  • Accounting as mnitoring views (analysis job only?)

Essential tools

Where do I get help?

  • As a site
  • As a user
  • User guide:
    • For example, how do I execute a job?
      • PanDA job
      • "Local" (batch)?
      • PROOF?
      • ...
  • aggregate documentation
  • Support model
  • DAST implications

More advanced use cases

  • Tier3-Tier3 data sharing
  • Big Tier3 description (including National Analysis Facilities)
    • HW, Software and policies

Major updates:
-- MassimoLamanna - 12-Feb-2010 -- DougBenjamin - 18-Sep-2010

%RESPONSIBLE% DougBenjamin
%REVIEW% Never reviewed

Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r1 - 2011-03-17 - AndrejFilipcic
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback