Central Harvester Instances

Technical documentation

For a technical description of Harvester components please visit the Harvester github wiki

Monitoring

Harvester machines

You can connect with the usual atlpan user
Name HarvesterID Description
aipanda170 cern_cloud Currently the main harvester machine submitting to Unified PQs, GCE, P1. It contains a local MySQL database and schedd.
aipanda171 CERN_aipanda171 Currently this machine submits to other main Grid PQs. In the future will share the load with aipanda170. The goal is to use a shared MySQL DB on demand (CERN IT hosted) and external schedd.

Important paths and files

Filename Description
/var/log/harvester All the harvester logs of the various agents
/usr/etc/panda/panda_harvester.cfg General configuration of subcomponents, DB connection, etc.
/usr/etc/panda/panda_queueconfig.json Queue configuration
/data/atlpan/harvester_common/ Condor sdf templates and other files needed by Harvester
/data/atlpan/harvester_worker_dir/ sdf file submitted to Condor for each job
/data/atlpan/condor_logs/ Condor and pilot logs for each job

Restarting Harvester

[root@]# /usr/etc/rc.d/init.d/panda_harvester-uwsgi reload

MySQL

The most important tables in the DB structure can be found here, although it's under constant evolution. The DB configuration can always be found in the panda-harvester.cfg file, but currently you can connect to them like this:

[aipanda170]# mysql -u harvester -p harvester [aipanda171]# mysql -h dbod-harv.cern.ch -P 5501 -u admin -p HARVESTER

You will need to know the password to connect. Run only queries you understand and where you know what you are doing.

Condor Schedd machines

These are external condor schedd nodes harvester nodes submit through. You can connect with the usual atlpan user
Name
aipanda023
aipanda024

Currently only aipanda024 submit new jobs to CE (aipanda023 under migration of job logs). In the future they will be interchangeable.

Restarting condor

[root@]# systemctl restart condor

Important paths and files

Filename Description
/etc/condor/ Condor configuration, our configuration usually goes under config.d
/var/log/condor/ Condor logs, one per each agent

Troubleshooting

Condor schedd related

more...

FahuiLin - 2018-07-20

Topic revision: r7 - 2018-09-14 - FernandoHaraldBarreiroMegino
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    PanDA All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback