T2TWDecX

December09 Exercise

The purposes of December '09 exercise are:
  • Challenge the readiness of T2_TW_Taiwan site.
  • Teach the twcms users to make good use of T2_TW_Taiwan resource.
  • Optimize the resource sharing with tw-atlas users.

  • Contact people: K.F Chen (NTU), C.M. Kuo (NCU), S.J. Zhou (AS), Yuan Chao(NTU), S.W. Li (NCU)
  • ASGC T2 contact: H.L. Shih, G. Qing.

The Exercise Details

  • DONE STEP0: Tutorial and grid role test
  • Work in progress, under construction STEP1: MC production and data subscription
  • Work in progress, under construction STEP2: Analysis ntuple production
  • Work in progress, under construction STEP3: Test with tw-atlas users
  • Work in progress, under construction STEP4: Review & summary

Pre-exercise preparation

  • Documentation web page: (Jason & Gang)
  • Monitoring web page (Jason & Gang)
    • CPU / Disk storage Ganglia page:
    • Job status
    • Total number of queues available. (Jason & Gang)

Two-hour tutorial one week before the exercise (STEP0) with one-hour lecture plus one-hour hand-on session.

  • Datasets: Official MC productions. (to be updated)

  • Useful links:

The "starting page" Twiki: http://wiki.twgrid.org/apwiki/test?action=show

FTT Batch System Running Job Monitoring (running): https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/ftt/ftt_batch_running.html

FTT: https://osadm.grid.sinica.edu.tw/acctmon/?bat=f-ce01&key=month&num=14

ASGC Batch System Running Job Monitoring (pending): https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/batch/asgc_batch_pending.html

ASGC Batch System Running Job Monitoring (running): https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/batch/asgc_batch.html

CASTOR2 LSF Job Submission Monitoring https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/castor2/platform_lsf.html

ASGC CASTOR Atlas Job Scheduling Monitoring https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/castor2/c2_lsf_atlas.html

CASTOR2 Network Throughput - CMS https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/castor2/castor2_cms_throughput.html

CASTOR2 Network Throughput - Atlas https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/castor2/castor2_atlas_throughput.html

ASGC RRD Gateway (castor2): https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/castor2/

ASGC RRD Gateway (ftt): https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/ftt/

Mailing list for user meeting: http://lists.grid.sinica.edu.tw/mailman/listinfo/user-meeting

MC Production & Data subscription (Dec. week I)

  • We can produce b' signal pythia/madgraph MC at this moment.
  • Background processes?
  • Output to FTT SE and publish in local DB

  • Samples to be subscribed to T2_TW
    • 7 TeV official MC?

  • Results:

  • Problems & solutions:

Analysis ntuple production

run analysis code to make ntuples from the MC production of the first week or the subscribed samples. We will try output the results to either CERN castor, T2 SE and T3 SE. (STEP2) We need some people using non-twcms role during this test and submit the jobs first. As we have very very limited slots for non-twcms, the jobs would just stay in queue, we can give the pressure on the batch system in this way. These jobs should last for at least two days and the tw-cms users start from the 2nd day.

1) fill all the queues from a generic cms user (non tw-cms) member; cut in test from tw-cms / cut in test from tw-atlas user. 2) fill all the queues from a generic atlas user (non tw-atlas) member; cut in test from tw-cms / cut in test from tw-atlas user. 3) fill all the queues from a tw-cms user; cut in test from a generic atlas user / cut in test from tw-atlas user. 4) fill all the queues from a tw-atlas user; cut in test from a generic cms user / cut in test from tw-cms user.

Co-test with tw-atlas users

people repeat the same test with swapped stage-out SE. Those who use non-twcms role should swap the role with others. And we have tw-atlas people submit the jobs at the same time. (STEP3)

  • Results:
  • Problems we met:

Review and Summary

  • How to improve the usage of T2_TW? (attached slides)
    • ychao_20091105.pdf: Test items and thoughts on how to improve the T2_TW usage experience

- One user meeting for summary and discussions - Few further tests for performance fine-tuning?

-- YuanChao - 12-Nov-2009

Topic attachments
I Attachment History Action Size Date Who Comment
PDFpdf ychao_20091105.pdf r1 manage 169.9 K 2009-11-12 - 12:19 YuanChao Test items and thoughts on how to improve the T2_TW usage experience
PDFpdf ychao_20091201.pdf r1 manage 2336.5 K 2009-12-01 - 16:45 YuanChao Yuan CHAO's slides for T2 usage tutorial
Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r5 - 2009-12-10 - YuanChao
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback