TWiki> LCG Web>RegistrationTFend (revision 9)EditAttachPDF

Detailed notes from the last VO Registration Task Force (TF) Workshop held at CERN Jan 22-26 2007

Monday Jan 22nd

Dedicated to VOMS Generic Attributes (GAs): The VOMS GA requirement was mentioned by LHCb during the last TF workshop (March 13-17 2006) and discussed in greater detail during the last TF checkpoint meeting (October 24 2006). The LHCb VO Admin, Joel Closier spent the day with the TF to ensure the LHCb VO requirements are well understood and correctly implemented, ready for use a.s.a.p. Action 1

LHCb desires a GA, which they call nickname to carry the value of the VO member's afs login to facilitate filenames in storage (DN usage is inconvenient in file paths that also contain slashes).

As Atlas seems to intend the GA to represent something completely different, related to funding, hence influencing job priority in the batch system queue at the site, the need to precisely record the Requirements' list was emphasised.

The developers discussed methods to improve communication, especially when there are any new feature proposals or changes in api. We had about 2 hours discussion on the topic of GAs first within the TF, understanding how the GA is present in the voms-proxy, how VOMRS would implement GAs or branch-off to voms-admin for GA entry and/or update. Other details we discussed for another 2 hours in the framework of the VOM(R)S BOF.

Monday Jan 22nd WLCG VOM(R)S BOF

Participants: TF (Andrea Ceccanti, Vincenzo Ciaschini, Tanya Levshina, Lanxin Ma, Ian Neilson), VO Admins (Andrea Sciaba [CMS], Joel Closier [LHCb], Alessandro de Salvo [ATLAS], Patricia Mendez Lorenzo [representing Alice, Geant4, Gear, UNOSAT], Jiri Chudoba [Auger], Tomas Kouba [VOCE, Auger]), JSPG (Dave Kelsey), EGEE TCG (Claudio Grandi), NDGF (Michael Gronager), VOMS Tester (Maria Alandes Pradillo), VOM(RS) new service manager (Remi Mollon), site managers (Alessandra Forti, Gidon Moont).

Conclusions: There are definitely two experiments that need Generic Attributes (Atlas and LHCb). CMS start evaluating the possibility to use GAs to grant access to web pages. Alice's only categorisation of users will be the use of the Group=lcgadmin. GAs are not needed as all the rest is done by AliEn. Unosat, Geant4, Gear use no voms-aware services, so they need no GAs. VOCE and Auger need no GAs either.

We have asked VO Admins to provide written requirements in this document until mid-February 2007. EGEE VOs will be asked by Cal Loomis (NA4) to contribute. Claudio passed this request to Cal.

It was made absolutely clear that GAs should be handled on the level of a VO-specific software and will not be interpreted by the grid middleware.

voms-admin and voms core have already a version that can handle GAs. This will be tested on their test installation at CNAF by LHCb. Bilateral agreement will handle the details. The elaborate requirements now being collected will help improving the current implementation to satisfy all needs.

VOMS-ADMIN will ensure:

  1. the GA uniqueness
  2. the possibility for bulk selection (list of all DNs for a specific GA)

VOMRS will implement the following minimal requirements:

  1. Provide the ability to use GAs (configurable)
  2. VO admin can assign value to predefined GA during member's approval or later, if changes needed to be done
  3. users will be notified about assigned GA and should be able to see it via vomrs
  4. GAs will be attached only to VO membership (will not be Group-related)

AOB:

1. VODB Replication: Jiri expressed interested in voms database (VODB) replication. Replication assumes one master and multiple replicas. This is mostly working for MySQL. To do:

  • Re-launch the CNAF-CERN Oracle teams' communication on testing the code that allows this Action 2
  • Open a savannah ticket for gridmap generation script change. It would consult the replicas if the master fails. Action 3

2. CA Rollover: The recent UK and INFN changes of the CA DNs (with no change of the user DNs) created a lot of confusion for users and VO Admins. A page of help is available on the GOC wiki about this.

3. End of the TF: The LCG Task Force, achieved its goals and should be dissolved. It was suggested that the GDB should advise on how to proceed and organise future communication between VO managers and voms/vomrs developers. By the time these notes are published this was done during the February 2007 GDB.

Tuesday Jan 23rd

jdk-1.5: After a few weeks of testing on voms-test.cern.ch, we installed jdk-1_5_0_09-linux-i586.rpm on the 3 CERN production servers voms10[1,2,3].cern.ch and we changed the site configuration files accordingly. No problem was presented ever since related to this java version change.

voms bugs: Maria Alandes prepared a document containing the VOMS bugs in savannah. The bugs were examined one by one. Some of them were fixed for quite some time but the relevant glite patch was not yet in production for us to install it (example: https://savannah.cern.ch/bugs/?13888 which is fixed months ago but the relevant patch #869 required persistent escalation to the EMT, in order to get, by the time these notes are written, to the production repository. Joachim advised us to always pursue via the EMT every urgent request.

WLCG VOMS Groups/Roles session attendance: The TF went to the Main Amphitheatre to attend this session. We could identify no request to the VOM(R)S developers for the use of Groups/Roles by the grid services. Here are the notes of that session by Remi:

   WLCG Workshop : VOM(R)S Groups/Roles - Jan, 23rd 2007
Andrea Sciaba (CERN) - Experiments
Jeff Tamplon (NIKHEF) - Sites
Chris Brew (RAL) - Tier2
Maarten Litmaath (CERN) - Deployment
& Convenor David Kelsey (RAL)

Andrea Sciaba : Usages of VOMS for the experiments
==================================================
To implement fine-grained management of permissions and privileges :
. job priorities
   - number of cpus for different activities
   - prioritization mechanisns
   - match making depending on VOMS-specific info
. Data Management
   - VOMS aware storage and catalogues
      + quota, acl, ...
   - VOMS aware transfer tools
      + prioritization depending on groups/roles
   - Now
      + VOMS ACLs in LFC & DPM
      + only primary group is taken into account
   - Future
      + all user groups should be taken into account
      + VOMS support in all SRM systems
. Software installation
   - special groups/roles to install VO software at sites
. VO-specific services
   - roles to service admin to change service configuration
. Accounting related to VOMS FQAN (not only VO)
. VOMS group tree
   - regional groups
   - activity related groups

Jeff Tamplon : site perspective
===============================
. Where we are :
   - VOMS work for SW installation
   - Elementary separation of storage
   - Web sites access via cert
. Where we are almost :
   - Basic job priorities
      + separation of shares via groups/roles
      + publishing of ERTs per groups/roles
      +match making using groups/roles
. Where we ain't :
   - Accounting via groups/roles
   - Information management : explosion of group/role combination ?
   - More flexible DM VOMSification (better than all or nothing !)
. Lots of decision to make
   - how to choose most appropriate group/role in case of multiple matches ?
   - how to make sure all subsystems make the same choice ?
   - how to limit damage if they don't ?
. Make sure mapping VOMS to unix doesn't create 'hidden' limitations

Chris Brew : A Tier2's concerns
===============================
. proposal for VOMS based scheduling
   - voms groups => unix groups => MAUI
. on CE
   - proliferation of pools accounts & groups
   - frequency of updates for maui.cfg
   - maintainability
. on SE
   - separate pools/endpoints for each groups ?

Maarten Litmaath : VOMS & Deployment
====================================
. schedules must be driven by EMT & TCG
   - experiments have to ensure the right issues are on agenda
. new yaim on CTB allows for special cases other than sgm and prd
   - allow new groups and roles to be mapped differently
   - none has been added so far
. job priorities WG batch system recipe
   - documentation
   - needs new torque+maui, in certification
   - needs new lcg-info-dynamic-scheduler rpms, in certification
   - needs changes in YAIM
. accounting for groups and roles almost ready
   - lcg-ce awaiting YAIM update
   - APEL patch expected in few days
. Data Management :
   - supported by lfc & dpm
   - supported by dCache to some extent, via gPlazma callout
   - not yet supported by castor, still using grid-mapfile
   - not seen as a priority for this year
   - FTS 2.0 supports voms, to be released in few weeks

Questions
======
> To Marteen : what in case of multi group VOMS proxy ?
=> in lfc/dpm only the primary group is taken into account

Wednesday Jan 24th

Oracle usage optimisation: The table on page VomsOracleImprove was used as reference material for our discussion.The Oracle group applies pressure for use of long connection strings, i.e. multiple listeners and OCI connections. The multiple listeners were introduced so far in /opt/glite/etc/voms/tnsnames.ora and /var/glite/etc/voms-admin/VOname/voms.database.properties by running the 'home-made' voms.massage script after the configuration script /opt/glite/etc/config/scripts/glite-voms-server-config.py. This was a confusing procedure. The new voms-admin and glite configuration script take care of properly defining all the listeners. We installed the patch containing [https://savannah.cern.ch/bugs/?17456][the fix]] in production. We understood that:

  • the voms core configuration files /opt/glite/etc/voms/VOname/voms.conf should contain the Oracle account_W to enjoy a big number of parallel connections.
  • the voms-admin configuration files /var/glite/etc/voms-admin/VOname/voms.database.properties cannot, unfortunately, contain the account_W because voms-admin always checks if there is something to re-configure, while starting. Either:
  • the voms-admin configuration scripts have to be changed in order to split the automatic upgrade from normal operations or
  • we continue using the account and live with the 10-sessions limit and the addiitional inconvenience of regular password expiration, which, so far, issues no warning. Action 5

Vomrs uses the vomrs_account_W in each VO configuration files but when an upgrade takes place, which involves change of the database schema the vomrs_account should be used instead, which are enabled to do CREATE_TABLES and other privileged commands.

Concerning the use of OCI, instead of thin connections to the database, progress was made with advice from Piotr Nyczyk. It must be completed a.s.a.p. to be moved to production. Action 4.

Miguel Anjo sent a SQLPlus tutorial to assist the developers: in html or in pdf. Other information: http://www.oracle.com/pls/db102/portal.portal_db?selected=4 These pages also accessible from http://oradoc.cern.ch/

There was a question on licence for use of the rpm built at CERN for Oracle instantclient can also be used by Tier1 centres. Dirk Duellmann, responsible for Physics Database services advised that the glite packaging people decide on a common way of handling this dependency across glite packages in the release. If they all agree we can (re-)discuss how to properly include the package with the licence forms. Markus Schulz and Alberto di Meglio were informed of this requirement. Action 6

Discussion of vom(r)s testing for Oracle patches and security updates:

  • Test voms/voms-admin inserts and other voms-admin commands submitted via automated bulk transactions, under heavy load, developed by Victor. Action 7.
  • multiple concurrent gridmap file generation processes. Action 8.
  • voms-proxy commands. A stress test script will be provided by Vincenzo. Action 9.
  • vomrs testing should contain Tanya's stress tests for inserts.

These are the links we have today about test suites and reports: * Voms test suite * voms-admin test suite * Vom(r)s test reports * Detailed Vom(r)s test reports The testers will make the necessary updates following this discussion. Action 10.

By the time these notes are written Maria D. created the mailing list project-voms-test@cern.ch for future information distribution that affects voms and requires testing.

Thursday Jan 25th

VOMRS bugs' discussion: The following list of tickets were Fixed and (by the time these notes are written) were Closed in https://savannah.cern.ch/projects/lcgoperation/

https://savannah.cern.ch/bugs/?12042, https://savannah.cern.ch/bugs/?14990, https://savannah.cern.ch/bugs/?15146, https://savannah.cern.ch/bugs/?15164, https://savannah.cern.ch/bugs/?15270, https://savannah.cern.ch/bugs/?4861, https://savannah.cern.ch/bugs/?10446, https://savannah.cern.ch/bugs/?14653, https://savannah.cern.ch/bugs/?14762, https://savannah.cern.ch/bugs/?15012, https://savannah.cern.ch/bugs/?15134, https://savannah.cern.ch/bugs/?15153, https://savannah.cern.ch/bugs/?15244, https://savannah.cern.ch/bugs/?15744, https://savannah.cern.ch/bugs/?15751, https://savannah.cern.ch/bugs/?15842, https://savannah.cern.ch/bugs/?16558, https://savannah.cern.ch/bugs/?16562, https://savannah.cern.ch/bugs/?16724, https://savannah.cern.ch/bugs/?17392, https://savannah.cern.ch/bugs/?14993, https://savannah.cern.ch/bugs/?17050, https://savannah.cern.ch/bugs/?22272, https://savannah.cern.ch/bugs/?22223, https://savannah.cern.ch/bugs/?18722, https://savannah.cern.ch/bugs/?18298, https://savannah.cern.ch/bugs/?18013, https://savannah.cern.ch/bugs/?18002, https://savannah.cern.ch/bugs/?17392.

A number of other bugs were updated, after the workshop by Lanxin and are attached to these notes. VOMRS new bugs and remaining ticket updates are done in savannah. Example: https://savannah.cern.ch/bugs/?23723.

Re-evaluation of the CERN vom(r)s servers' architecture:

Documentation review: - Joachim's guide

Friday Jan 26th

DTEAM VO Registration flow review and presentation to the CERN ROC managers: - Tanya's dteam slides

Action list

Number What Who When Status
1 Test voms-proxies with GAs LHCb VO members and VOMS developers at CNAF a.s.a.p. Joel will test with the CNAF voms installation and LHCb application Ganga
2 Voms Oracle replicas CERN-CNAF Oracle experts a.s.a.p. Maria D. spoke to Dirk Duellmann. She 'll send email the experts.
3 mkgridmap script handling replicas Maarten when VODB replication tests completed. Maria D. spoke with Maarten. She'll open savannah tickets
4 Complete VOM(R)S re-packaging for OCI use Developers/Integrators/Deployers a.s.a.p. Andrea documented the method here. Joachim should write in the ticket what/when changes will appear in the glite-VOMS_oracle rpm. Tanya should update bug #19690 and bug #19692.
5 Decide on voms-admin config. change account_W Remi a.s.a.p. To open ticket and assign to Andrea
6 Make available the Oracle instantclient without licensing problems Joachim a.s.a.p. To open ticket and follow-up
7 Create _W accounts for Maria Alandes and Lanxin in int3r for performance/load testing. Miguel a.s.a.p.
8 add timing info to voms/voms-admin test suites. Victor a.s.a.p. Maria A. please open savannah ticket to use as reference
9 voms-proxy commands' stress test script. Vincenzo a.s.a.p. Maria A. please open savannah ticket to use as reference
10 Update test suites' definition to include Oracle 3-monthly updates' tests. Maria A. - Lanxin a.s.a.p.

-- Main.dimou - 06 Feb 2007

Topic attachments
I Attachment History Action Size Date Who Comment
Microsoft Word filedoc vomrs-bugs_Feb2007.doc r1 manage 120.5 K 2007-02-12 - 17:14 UnknownUser VOMRS bugs in February 2007 and comments
Edit | Attach | Watch | Print version | History: r13 | r11 < r10 < r9 < r8 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r9 - 2007-02-12 - MariaDimou
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback