YAIM logbook

September 2007

Wednesday 5th September

• GGUS tickets support
• Savannah tickets support
• External sites support for YAIM 4.0.0 certification
• preparing modules for YAIM 4.0.1: torque utils, e2emonit

Tuesday 4th September

• GGUS tickets support
• Savannah tickets support
• External sites support for YAIM 4.0.0 certification
• preparing modules for YAIM 4.0.1: se classic

Monday 3rd September

• GGUS tickets support
• Savannah tickets support
• External sites support for YAIM 4.0.0 certification
• preparing modules for YAIM 4.0.1: monbox

August 2007

Friday 31th August

• lcg CE module. Fixed problem with node-info.d file called lcg-ce.
• Started to check environment functions. Summary updated in task #5474.

Thursday 30th August

• Move some bugs from 1238 to 1239 as detected by Nuno that they affect only 3.1.
• Started to prepare 4.0.1 release. Adding headers into functions and checking new modules (BDII and clients).

Wednesday 29th August

• Branch and HEAD merge for core (cont). The same for clients. Changes also in new modules for se_classic and mon.
• Nuno has to verify the bugs of patch 1238 and 1239.
• Decided that external sites will test upgrade of testbed together with Louis when he is back from holidays.
• Tests with lcg CE installation and SGE server (support for Goncalo)

Tuesday 28th August

• Modified config_wn to fix bug #28082 and #27896
• Branch and HEAD merge for core.
• Discussion about _check and _setenv functions. Wiki updated.
• glite-yaim-clients-4.0.0-5.noarch.rpm generated because of a bug fix released in core and not in clients.

Monday 27th August

• Corrected Makefiles for yaim modules and wiki updated.
• Mail with YAIM directory structure and wiki updated.
• Fixed yaim core to be able to configure 3.0 and 3.1 LFC.
• VOBOX fix to configure one single sgm account.
• YAIM leaflet first version.
• Generated RPMs for YAIM 4.0.0:
  • glite-yaim-core-4.0.0-11.noarch.rpm
  • glite-yaim-clients-4.0.0-4.noarch.rpm

Friday 24th August

• Changes for Di. Going back to GRIS in the lcg CE.
• 3.0 and 3.1 configuration of LFC in yaim 4.0.0.
• Fix WN TAR in 3.0 that didn't contain the _30 suffix.
• Corrected version of CHEP poster with Markus comments.
• Generated RPMs for YAIM 4.0.0:
  • glite-yaim-clients-4.0.0-3.noarch.rpm
  • glite-yaim-core-4.0.0-10.noarch.rpm
  • glite-yaim-lfc-4.0.0-4.noarch.rpm
• New certification wiki for third phase of YAIM 4.0.0 certification.

Thursday 23rd August

• Created TORQUE modules in yaim and support for SGE.
• VOBOX 3.1 configuration, first review.
• Changes for Di. Going back to GRIS in the lcg CE.
• Meeting with Oliver, Gergo and Di.

Wednesday 22nd August

• New yaim modules org.glite.yaim.se-classic, org.glite.yaim.rb.
• Sencond version of CHEP poster.
• First version of yaim core 4.0.1-1 for BDII 3.1 testing. Problems to be fixed by Gergo.
• Improving YAIM guide with Antun Balaz comments.
• Improving Developers guide and YAIM guide due to 3.0 and 3.1 configurations in same module.

Tuesday 21st August

• org.glite.yaim.bdii with 3.1 BDII configuration now created.
• Splitting YAIM org.glite.yaim.lcg-ce and org.glite.yaim.glite-ce
• Problems with LFC, 3.0 and 3.1 configuration has been mixed up.

Monday 20th August

• CHEP poster together with Gergo to see general structure. Finished my part.
• GGUS tickets
• Reconfiguration of nodes in 3.1.1 YAIM.

Friday 17th August

• Branching and recreating rpms with correct spec files:
  • glite-yaim-core-4.0.0-9.noarch.rpm
  • glite-yaim-clients-4.0.0-2.noarch.rpm
  • glite-yaim-dcache-4.0.0-2.noarch.rpm
  • glite-yaim-dpm-4.0.0-2.noarch.rpm
  • glite-yaim-fts-4.0.0-5.noarch.rpm
  • glite-yaim-lfc-4.0.0-3.noarch.rpm
  • glite-yaim-myproxy-4.0.0-2.noarch.rpm
• Versions table in YAIM planning page.
• Batch system information in YAIM guide.
• New YAIM certification page created for second phase of YAIM 4.0.0.
• Discussion with Maarten and Dave Kant about APEL configuration and information system detection.
• Statistics information on how long it takes to develop, certifiy, move to PPS and to prod a YAIM release.

Thursday 16th August

• Patch #1283 created. Following rpms have been generated:
  • glite-yaim-core-3.1.1-9.noarch.rpm
  • glite-yaim-core-3.1.1-5.noarch.rpm
• Fix for gip and bdii config functions ready. Following rpms have been generated:
  • glite-yaim-core-4.0.0-6.noarch.rpm
• But we decide that WMS 3.0 should stay the same as far as gip config is concerned. I also remove the config_apel_blah function.
  • glite-yaim-core-4.0.0-7.noarch.rpm
• I decide to update all the Makefiles and spec files:
  • glite-yaim-clients-4.0.0-2.noarch.rpm
  • glite-yaim-dcache-4.0.0-2.noarch.rpm
• However, I realise I need to branch clients and dpm because there are changes in head. I need to repeat this.
• APEL problem detected by Ulrich. I think this is not a bug but probably it's due to LSF that it's not configured by YAIM.

Wednesday 15th August

• Information system problem in yaim 4.0.0 now clear. Implement changes in CVS.
• Gergo has implemented fix for cron job problem and termination of yaim script. They are in the corresponding branches. I can generate now rpms.
• MPI support fot YAIM developers. Module created in ETICS.

Friday 10th August

• VO WG comments. Update of YAIM guide.
• Support for MPI yaim developer.
• Branch glite-yaim-dpm_branch_3_1_1_4 and glite-yaim-core_branch_3_1_1_8 created to implement fix for Bug #28459
• Branch glite-yaim-core_branch_4_0_0_5 to implement fixes for YAIM 4.0.0 bugs reported by Di.

Thursday 9th August

• WMS and LB rpms generated. RPM generation is now Di's task.
• Savannah bugs review
• VO WG reply to manual steps for removing a VO in YAIM.

Wednesday 8th August

• GGUS tickets review and Savannah tickets review
• WMS and LB rpm generation for the new WMS. Problems with Makefiles and spec files.

Tuesday 7th August

• GGUS tickets review and Savannah tickets review

Monday 6th August

• Study information system configuration changes from YAIM 3.0.1, 3.1.1 and 4.0.0
• GGUS tickets review

Friday 3rd August

• GGUS tickets review - Specially VO WG tickets and its report
• 3.1 DPM -> changes in the configuration by Louis (mysql daemon)
• LFC 3.1 -> Oliver working on it. Different from 3.0 configuration.

Thursday 2nd August

• Meeting with Oliver. Check the minutes.
• VO WG report. YAIM improvements.
• GGUS tickets review.

Wednesday 1st August

• Study config_mkgridmap and pool accounts. FIO has installed yaim-core 4.0.0-5! This is solved by using their old users.conf with static accounts
• GGUS tickets
• Splitting yaim core - remaining components.

July 2007

Tuesday 31st July

• Ask Steve to create his own Makefile and specfile like in yaim-core. He found problems. We need to check with Laurence.
• Problems with XMLHelper.py that doesn't work. Thinking of doing something else simpler.
• FIO problems with sgm/prd pool accounts.
• c-shell enviroment critical bug #28317
• Set up lcg CE to find out why gatekeeper does not start

Monday 30th July

• Di still testing yaim 4.0.0. Problems found with ldif parsing as explained in YAIM certification wiki pages.
• New variable needed in site-info.def for SITE_BDII_HOST. New core rpm is needed.
• New FTS rpm is needed to include Gavin's fix for FTS. Also ed dependency required by Steve for FTM.

Tuesday 24th July

• Contact Steve to see what to do with FTM module.
• CVS access for AMGA module. One developer doesn't have AFS account. Same for another collaborator for WMS/LB.
• Review wiki pages and update them when necessary.
• Problem with gip and glite CE. Modifications needed in node-info.def and config_gip_old. New core rpm is required. glite-yaim-core-4.0.0-5.

Monday 23rd July

• Meeting with Oliver: not many outstanding problems with yaim 3.1.1 which is still in PPS. Probably won't go to production.
• Di is testing yaim 4.0.0
• How to split yaim in more modules, which functions appear only in one module? On going...

Thursday 12th July

• Finish testing APEL patch
• Finish fixes for YAIM.
• Created rpms for YAIM 4 and updated Certification wiki:
  • glite-yaim-clients-4.0.0-1.noarch.rpm
  • glite-yaim-core-4.0.0-1.noarch.rpm
  • glite-yaim-dcache-4.0.0-1.noarch.rpm
  • glite-yaim-dpm-4.0.0-1.noarch.rpm
  • glite-yaim-fts-4.0.0-1.noarch.rpm
  • glite-yaim-lb-4.0.0-1.noarch.rpm
  • glite-yaim-lfc-4.0.0-1.noarch.rpm
  • glite-yaim-myproxy-4.0.0-1.noarch.rpm
  • glite-yaim-wms-4.0.0-1.noarch.rpm
• Implemented Rolanda's fix for bugs #27984 and #25667.
• New version of users.conf provided by Maarten.
• re-created rpms for glite-yaim-core to include Rolanda's fix and Maartens new users.conf:
  • glite-yaim-core-4.0.0-2.noarch.rpm
  • glite-yaim-core-4.0.0-3.noarch.rpm

Wednesday 11th July

• Finished fixing APEL and UI bugs
• Laurence has tested GIP config changs with Ulrich.
• MIP module created in cvs. Need to create module in etics.
• Andreas has finished his fixes.
• Gavin has committed also his fixes and wants us to generated the rpm.

Tuesday 10th July

• Setting up virtual testbed for testing configuration. Several problems with job submission due to lcg CE misconfiguration.
• testing APEL and fixing configuration problems.
• Laurence has finished his fixes and wants to produce rpm to test them.
• AMGA module created for Asterios. CVS and etics.

Monday 9th July

• Clients rpm regenerated because mistake in WN_torque node-info.d file name. To include in the repo:
  • glite-yaim-clients-3.1.1-8.noarch.rpm
• After talking to Oliver, this is the list of bugs/patches for the next YAIM release:

Savannah bug/patch	Description	Affected yaim module	Developer	Priority	Status
#27761	gatekeeper restarted	core	Maria	High	Done
#27600	glite-UI config problems	clients	Maria	Medium	Done
#1164	APEL	core	Maria	High	Done
#1093	RGMA	mon	Laurence	Medium	Done
#1185	GRIS + lcg CE	core	Laurence	Medium	Done
#894	Rebranded GIP that includes inproved LDIF parsing	core	Laurence	High	Done
#27437	lcgenv	core	Andreas	High	Done
#27533	Glue schema	core	Laurence	Low	Done
#27694	static accounts in gridmapdir	core	Maarten	High	Done
#27816	WN_TAR	clients	Andreas	High	Done
#27845	Problem with 3.0 WN torque installation	clients or core?	Maria/Di	High	To be tested

• The following list can wait:

Savannah bug/patch	Description	Affected yaim module	Developer
#23746	multiple RBs and WMSs	core?	Gergo
#26757	subclusters	core	need talk to Steve
#1173	Blah and gLite CE	glite CE	Maria

• YAIM 4 is the name of the next YAIM version.
• Planning and Certification page updated for the new YAIM 4.
• YAIM 3.1.1 is now certified.
• Maarten has been asked to provide good examples for users.conf.

Friday 6th July

• Include core, clients and dpm rpms in the repo:
  • glite-yaim-core-3.1.1-7.noarch.rpm
  • glite-yaim-clients-3.1.1-7.noarch.rpm
  • glite-yaim-dpm-3.1.1-4.noarch.rpm
• Makefile problem in core that needs a new rpm. Include in the repo:
  • glite-yaim-core-3.1.1-8.noarch.rpm
• Latest Di's tests are fine. Waiting for Louis to upgrade testbed with latest affected rpms.
• DENY tags will be tested but not included in the YAIM release. There is a function called config_gip_with.DENY under /afs/cern.ch/project/gd/www/yaim/testing/ that can be used for testing.

• TO DO:
  • The following bugs are in the queue to be implemented for the next yaim release:

Savannah bug/patch	Description	Affected yaim module	Developer
#27761	gatekeeper restarted	core	Maria
#27600	glite-UI config problems	clients	Maria
#1164	APEL	core	Maria
#1093	RGMA	mon	ask Laurence
#1185	GRIS + lcg CE	core	ask Laurence
#894	Rebranded GIP that includes inproved LDIF parsing	core	ask Laurence
#26757	subclusters	core	need talk to Steve
#27437	lcgenv	core	Andreas
#27533	Glue schema	core	ask Laurence
#27694	static accounts in gridmapdir	core	ask Maarten
#27816	WN_TAR	clients	Andreas
#23746	multiple RBs and WMSs	core?	Gergo
#1173	Blah and gLite CE	glite CE	Maria

Thursday 5th July

• Meeting with Oliver and Gergo in the morning.
• TAR UI and TAR WN for 3.0 doesn't work with yaim. Problem with yaim logfile as well. Problems fixed and clients rpm regenerated.
• Gergo fixes WMS and DPM problems.
• Dependencies fixed for WMSLB, LB and WMS in 3.0 repository.
• LB old configuration reintroduced in yaim core.
• Laurence fixes the problems with the site BDII configuration.
• core and dpm rpms regenerated.

Wednesday 4th July

• WMS problem found by Louis in bug #27733
• DPM problem found by Louis in bug #27731
• Di founds out that Site BDII configuration doesn't work. We realise Laurence has committed a new config_gip_only that it's called by the site BDII.
• CIC portal meeting with Dimitar, Maria, Maite and Nick. YAIM will still has to provide good and real examples.
• I created SGE modules in CVS

Tuesday 3rd July

• YAIM core regenerated to include fix for bug #27539
• Include fts, myproxy and core rpms in the repo
  • glite-yaim-core-3.1.1-6.noarch.rpm
  • glite-yaim-fts-3.1.1-4.noarch.rpm
  • org.glite.yaim.myproxy-3.1.1-4.noarch.rpm
• Check with Maria Dimou GGUS tickets. She will provide a link to our GGUS tickets.
• My GGUS support access is fixed and I can edi now GGUS tickets.
• Decision that certification of yaim should be first done in conf VMs with rpms in /afs/cern.ch/project/gd/www/yaim/, then move to upgrade of the ctb testbed + external sites with rpms in the cert repo and then move to PPS if everything is fine.

Monday 2nd July

• Discussion with Gergo on Job Priorities WG meeting and how to implement the fix in YAIM.
• Update Bug #25693 with the decision taken on the Job Priorities WG.
• Send mail to the Job Priorities WG mail to ask how certification of DENY tags should be done.
• Laurence points out bug with Glue schema and slc4. This won't go in the first YAIM 3.1.1 release.
• Update of YAIM certification page with latest Di's reports.
• Gergo and Andreas point out important fix for grid-env.sh that should go in first YAIM 3.1.1 release. Reported in bug #27539
• Di finds out that myproxy is missing node-info.d file.

• TO DO:
  • Regenerate myproxy rpm and yaim core rpm.
  • Ask Oliver what we do with gridice support on the glite CE (Di found some errors due to this).

June 2007

Friday, 29th June

• Update documentation pages (YAIM developers guide, YAIM certification page with Di's results...)
• Job priorities WG meeting

Thursday, 28th June

• Meeting with Oliver and Gergo. Check the minutes.
• Created Bug #27600 to keep track of 3.0 glite-UI issues found by Di.
• Steve Traylen finds out that we haven't included support for FTS 1 in the yaim FTS rpm. Gergo generates the new rpm containing also FTS 1.
• Update documentation pages (YAIM developers guide, YAIM certification page with Di's results...)

Wednesday, 27th June

• All the generated rpms are wrong because they depend on glite-yaim-core-HEAD. This configuration still contains the provides glite-yaim that we don't want.
• We modify all the ETICS configurations to point to the good glite-yaim-core configuration which is glite-yaim-core_R_3_1_1_1.
• local and remote build are ok. Juha and Joachim contacted since the new rpms are ready.
• RPMs generated:
  • glite-yaim-core-3.1.1-5.noarch.rpm
  • glite-yaim-clients-3.1.1-4.noarch.rpm
  • glite-yaim-dpm-3.1.1-3.noarch.rpm
  • glite-yaim-lfc-3.1.1-3.noarch.rpm testing
  • glite-yaim-fts-3.1.1-3.noarch.rpm
  • glite-yaim-wms-3.1.2-2.noarch.rpm
  • glite-yaim-lb-3.1.1-2.noarch.rpm
  • org.glite.yaim.dcache-3.1.1-2.noarch.rpm
• Juha creates the 3.0 repository.
• After checking installation of 3.0 WN, SA3 is contacted again to tell them that they can start testing glite 3.0 repository.
• site-info-vm.def for Di to use the VM to start testing yaim.
• YAIM 3.1.1 guide in the twiki.
• Certification really starts

• TO DO:
  • Update documentation in the wiki
  • Plan what's next

Tuesday, 26th June

• I check glite CE configuration but it fails because it contains config-glite-ce-glite_3.0 instead of config-glite-ce-old (in the function list). Similar thing for WMS. I fix this. YAIM 3.1.1 configures gLite CE. We can update the documentation and say we will support 3.0 gLite CE and WMS (I haven't tested WMS because Nuno did it some days ago and it worked).
• We remove LB from yaim core completely and glite-bdii from node-info.d.
• ETICS makes yaim modules obsolete and provide glite-yaim. That is because they depend on yaim core which obsoletes and provides glite-yaim. We realise that we don't need to provide glite-yaim with yaim core, since all the metapackages dependencies are going to be updated. We can create the rpms without having to wait for ETICS fix.
• External sites are contacted to wait to start testing since we need to regenerate the rpms.
• We need to create the dcache and the lb yaim rpms. SGE won't be ready for this release.
• We tag CVS and ETICS configuration for all yaim modules. ETICS is very slow and it's not easy to find out how to create dependencies. We find a problem with:
  • dcache module, whose rpm is generated as org.glite.yaim.dcache instead of glite-yaim-dcache.
  • remote build to create the rpms for 3.1 repository. Alberto is going to create the rpms for us.
• Since some of the _30 functions are very similar to the 3.1 version, Gergo merges them but forgets to upgrade some function lists. The clients rpm needs to be regenerated.
• Maarten's fixes are committed.
• I test gLite CE, BDII, WN and UI (slc3 and slc4), WMS:
  • The WMS repository has broken dependencies
  • The WN in SLC4 fails due to jpackage dependencies
• RPMs generated:
  • glite-yaim-core-3.1.1-4.noarch.rpm
  • glite-yaim-clients-3.1.1-3.noarch.rpm
  • glite-yaim-dpm-3.1.1-2.noarch.rpm
  • glite-yaim-lfc-3.1.1-2.noarch.rpm testing
  • glite-yaim-fts-3.1.1-2.noarch.rpm
  • glite-yaim-wms-3.1.1-2.noarch.rpm
  • glite-yaim-lb-3.1.1-1.noarch.rpm
  • org.glite.yaim.dcache-3.1.1-1.noarch.rpm
• Patch 1209 is updated with the latest rpms.
• I can't update Patch 1210 since the remote build failed.

• TO DO:
  • Tell Joachim to update glite 3.0 repository
  • Update Patch 1210 with ETICS repository rpms
  • Tell Juha to update glite 3.1 repository
  • Create 3.1.1 installation guide
  • Check certification wiki is up to date.

Monday, 25th June

• APT repository created in the morning by Juha. The metapackages dependencies are not changed and it's useless using APT to install. I clarify this in the YAIM Certification page.
• YUM repository not created yet. I need to do a remote build to create the rpms. ETICS doesn't work for the remote build. Mail sent to etics-support.
• External sites have been contacted. Certification matrix summarises what needs to be tested and who will test it.
• Gergo realises dependencies are not properly defined. glite-yaim-core obsoletes and provides glite-yaim. All the other modules should only provide themselves. This is not what we generated on the RPMs on friday. Gergo has problems to change this in ETICS and opens a bug. We need to regenerate the RPMs and set new dependencies.
• We decide to include small fixes reported by Maarten since we have to regenerate the RPMs.
• We haven't created the dcache RPM and it's already a splitted module.
• We decide we will support gLite CE and WMS, since it is distributed in 3.1.1 core already. I need to change references in the wiki and savannah where it says the opposite.
• I haven't tested gLite CE and WMS yet.
• We have to say Juha and Joachim that gLite CE and WMS should depend on yaim. We have forgotten about the SE classic and dcache as well.

• TO DO:
  • Fix dependency problems between yaim core, yaim modules and glite-yaim
  • Code Maarten's changes
  • Generate dcache rpm
  • Check gLite CE and WMS configuration with yaim 3.1.1 ---> After all these items, regenerate the RPMs and do not touch them unless we find the super bug.

• Number of certified metapackages = 0 as far as i know.

Friday, 22nd June

• RPMs generated:
  • glite-yaim-core-3.1.1-3.11.noarch.rpm
  • glite-yaim-clients-3.1.1-1.5.noarch.rpm
  • glite-yaim-dpm-3.1.1-1.1.noarch.rpm
  • glite-yaim-lfc-3.1.1-1.3.noarch.rpm testing
  • glite-yaim-fts-3.1.1-1.3.noarch.rpm
  • glite-yaim-wms-3.1.1-1.2.noarch.rpm
• Patch 1209 created for gLite 3.0 repository
• Patch 1210 created for gLite 3.1 repository
• Missing tests: gLite CE, old WMS (although Nuno tested it some time ago and it was ok), TAR WN, SE classic, dcache
• We decide not to support gLite CE and WMS

Pre-certification

• Removing some parts of the platform detection code - Gergo
• Checking platform detection code that it's able to execute function_glite3.0 or function_glite3.1 (criticial for UI and WN) - Gergo
• What to do with Python + XML.
  • Are we finally supporting CE in this release? Anyway, we can always release python and test it for the CE.
  • Are we going to release only bash scripts for CE and WMS?
  • If we release bash scripts, are we also keeping python + XML?

• vo.d/ code relies on get_vo_params and similar functions. I'm not sure all the functions work with this. We have to test it.
• services that are not separated from core should be moved to the specific module (ex. RGMA) - NO for 3.1.1. For the first release, these are the modules that are separated:
  • core
  • clients
  • LFC
  • DPM
  • FTS
  • CE
  • WMS

• SLC4 specific code for UI and WN has to be identified and integrated into yaim 3.1.1 - Maria to ask Andreas and Di. Di says that all the functions are different. Maria is now investigating 3.1.0 to compare.

-- MariaALANDESPRADILLO - 12 Jun 2007