TWiki>
PanDA Web>AtlasDistributedComputing>PanDA>BigPanDARPMbuildSystem>BigPanDARPMbuildManual (revision 1)EditAttachPDF
PanDA Web>AtlasDistributedComputing>PanDA>BigPanDARPMbuildSystem>BigPanDARPMbuildManual (revision 1)Manual build of RPMs for Big PanDA
Introduction
- This page is evolving. Please keep checking.
- This page contains information about manual build of RPMs for BigPanDA. Keep checking BigPanDARPMbuildSystem.
- Feel free to contact JaroslavaSchovancova and TorreWenaus with questions.
Manual build of ATLAS BigPanDAmon RPMs
- Let us assume that we need to build both
bigpandamon-atlasandbigpandamon-coreRPMs.
Manual build of bigpandamon-atlas
Update RPM configs in the devel branch
- Check the latest version available, do not forget to increment version string.
- cd to the local copy of svn+ssh://svn.cern.ch/reps/panda/panda-bigmon/trunk/bigpandamon-atlas
# cd panda-bigmon/trunk/bigpandamon-atlas
- edit file
branches/devel/bigpandamon-atlas/setup.cfg- update "requires" if necessary
# svn diff setup.cfg Index: setup.cfg =================================================================== --- setup.cfg (revision 18781) +++ setup.cfg (working copy) @@ -4,5 +4,6 @@ provides = bigpandamon-atlas release = 1 packager = Panda Team <hn-atlas-panda-pathena@cern.ch> -requires = python bigpandamon-core +requires = python + bigpandamon-core >= 0.0.4
- update "requires" if necessary
- edit file
branches/devel/bigpandamon-atlas/setup.py- update/check whatever is marked with
#FIXME, i.e. release_version, expected_extensions, packages, data_files
- update/check whatever is marked with
- commit changes to RPM configs
# svn ci -m "bigpandamon-atlas/branches/devel: prepared RPM setup."
Merge devel branch to trunk
- cd to the local copy of svn+ssh://svn.cern.ch/reps/panda/panda-bigmon/trunk/bigpandamon-atlas
# cd panda-bigmon/trunk/bigpandamon-atlas
- update local copy of trunk of bigpandamon-atlas
# svn up
- merge
branches/devel/bigpandamon-atlasbranch totrunk/bigpandamon-atlas- postpone conflicts for setup.py and setup.cfg
# svn merge --reintegrate svn+ssh://svn.cern.ch/reps/panda/panda-bigmon/branches/devel/bigpandamon-atlas --- Merging differences between repository URLs into '.': U atlas/prodtask/views.py U atlas/prodtask/request_views.py U atlas/prodtask/urls.py U atlas/prodtask/forms.py U atlas/prodtask/templates/prodtask/_index.html U atlas/prodtask/templates/prodtask/_reqdatatable.html C setup.cfg C setup.py
- resolve conflicts for setup.py and setup.cfg
# svn resolve --accept theirs-full setup.cfg Resolved conflicted state of 'setup.cfg' # svn resolve --accept theirs-full setup.py Resolved conflicted state of 'setup.py'
- postpone conflicts for setup.py and setup.cfg
- now go through changes before committing the merge, and adapt what is necessary, resolve collisions etc.
- diff from svnweb can be useful as an additional check: e.g. https://svnweb.cern.ch/trac/panda/changeset?new=18778%40panda-bigmon%2Fbranches%2Fdevel&old=18772%40panda-bigmon%2Ftrunk
- diff from svnweb can be useful as an additional check: e.g. https://svnweb.cern.ch/trac/panda/changeset?new=18778%40panda-bigmon%2Fbranches%2Fdevel&old=18772%40panda-bigmon%2Ftrunk
- commit the merge
# svn ci -m "Merged devel/bigpandamon-atlas to trunk/bigpandamon-atlas, RC for bigpandamon-atlas-0.0.4"
Create tag of bigpandamon-atlas-0.0.4
- cd to the local copy of svn+ssh://svn.cern.ch/reps/panda/panda-bigmon/trunk/bigpandamon-atlas
# cd panda-bigmon/trunk/bigpandamon-atlas
- create a new tag
svn copy \ svn+ssh://svn.cern.ch/reps/panda/panda-bigmon/trunk/bigpandamon-atlas \ svn+ssh://svn.cern.ch/reps/panda/panda-bigmon/tags/bigpandamon-atlas/bigpandamon-atlas-0.0.4 \ -m "bigpandamon-atlas-0.0.4"
Build RPM
- login to aipanda004 as jschovan/sbelov
# [lxvoadm]# ssh jschovan@aipanda004
- cd to a workdir
# MYWORKDIR="/data/jschovan/new-rpm/MYWORKDIR" # mkdir -p $MYWORKDIR # cd $MYWORKDIR
- prepare .rpmmacros
# echo "%_unpackaged_files_terminate_build 0" >> ~/.rpmmacros
- checkout the tagged code
# svn co svn+ssh://svn.cern.ch/reps/panda/panda-bigmon/tags/bigpandamon-atlas/bigpandamon-atlas-0.0.4
- cd to the tagged directory
# cd $MYWORKDIR/bigpandamon-atlas-0.0.4 # ls $MYWORKDIR/bigpandamon-atlas-0.0.4 atlas INSTALL.txt MANIFEST.in README.txt setup.cfg setup.py
- build RPM of bigpandamon-atlas-0.0.4
# QA_SKIP_BUILD_ROOT=1 python setup.py bdist_rpm ... lots of output, and then ... moving build/bdist.linux-x86_64/rpm/SRPMS/bigpandamon-atlas-0.0.4-1.src.rpm -> dist moving build/bdist.linux-x86_64/rpm/RPMS/noarch/bigpandamon-atlas-0.0.4-1.noarch.rpm -> dist # ls $MYWORKDIR/bigpandamon-atlas-0.0.4 atlas build dist INSTALL.txt MANIFEST MANIFEST.in README.txt setup.cfg setup.py # ls $MYWORKDIR/bigpandamon-atlas-0.0.4/dist -1 bigpandamon-atlas-0.0.4-1.noarch.rpm bigpandamon-atlas-0.0.4-1.src.rpm bigpandamon-atlas-0.0.4.tar.gz
- check files in the noarch RPM
- make sure that none of the python modules is missing (make sure every python module has init.py committed in SVN)
- make sure that none of the templates files is missing
- make sure that the static files are not missing
# rpm -qpl dist/bigpandamon-atlas-0.0.4-1.noarch.rpm | less
- remove .rpmmacros
# rm ~/.rpmmacros
Copy RPMs to the repository
- sudo su on aipanda004
- cd to the
distdirectory# MYWORKDIR="/data/jschovan/new-rpm/MYWORKDIR" # cd $MYWORKDIR/bigpandamon-atlas-0.0.4/dist
- cp src.rpm to /data/build/SRPMS
# cp bigpandamon-atlas-0.0.4-1.src.rpm /data/build/SRPMS/
- cp noarch.rpm to /data/build/x86_64
# cp bigpandamon-atlas-0.0.4-1.noarch.rpm /data/build/x86_64
Manual build of bigpandamon-core for ATLAS
Update RPM configs in the devel branch
- Check the latest version available, do not forget to increment version string.
- cd to the local copy of svn+ssh://svn.cern.ch/reps/panda/panda-bigmon/trunk/bigpandamon-core
# cd panda-bigmon/trunk/bigpandamon-core
- edit file
branches/devel/bigpandamon-core/setup.py- update/check whatever is marked with
#FIXME, i.e. release_version, expected_extensions, packages, data_files
- update/check whatever is marked with
- commit changes to RPM configs
# svn ci -m "bigpandamon-core/branches/devel: prepared RPM setup."
Merge devel branch to trunk
- cd to the local copy of svn+ssh://svn.cern.ch/reps/panda/panda-bigmon/trunk/bigpandamon-core
# cd panda-bigmon/trunk/bigpandamon-core
- update local copy of trunk of bigpandamon-core
# svn up
- merge
branches/devel/bigpandamon-corebranch totrunk/bigpandamon-core- postpone conflicts for setup.py and setup.cfg
# svn merge --reintegrate svn+ssh://svn.cern.ch/reps/panda/panda-bigmon/branches/devel/bigpandamon-core --- Merging differences between repository URLs into '.': ... updates ... C setup.py
- resolve conflicts for setup.py (and setup.cfg)
# svn resolve --accept theirs-full setup.py Resolved conflicted state of 'setup.py'
- postpone conflicts for setup.py and setup.cfg
- now go through changes before committing the merge, and adapt what is necessary, resolve collisions etc.
- diff from svnweb can be useful as an additional check: e.g. https://svnweb.cern.ch/trac/panda/changeset?new=18778%40panda-bigmon%2Fbranches%2Fdevel&old=18772%40panda-bigmon%2Ftrunk
- diff from svnweb can be useful as an additional check: e.g. https://svnweb.cern.ch/trac/panda/changeset?new=18778%40panda-bigmon%2Fbranches%2Fdevel&old=18772%40panda-bigmon%2Ftrunk
- commit the merge
# svn ci -m "Merged devel/bigpandamon-core to trunk/bigpandamon-core, RC for bigpandamon-core-0.0.4"
Create tag of bigpandamon-core-0.0.4
- cd to the local copy of svn+ssh://svn.cern.ch/reps/panda/panda-bigmon/trunk/bigpandamon-core
# cd panda-bigmon/trunk/bigpandamon-core
- create a new tag
svn copy \ svn+ssh://svn.cern.ch/reps/panda/panda-bigmon/trunk/bigpandamon-core \ svn+ssh://svn.cern.ch/reps/panda/panda-bigmon/tags/bigpandamon-core/bigpandamon-core-0.0.4 \ -m "bigpandamon-core-0.0.4"
Build RPM
- login to aipanda004 as jschovan/sbelov
# [lxvoadm]# ssh jschovan@aipanda004
- cd to a workdir
# MYWORKDIR="/data/jschovan/new-rpm/MYWORKDIR" # mkdir -p $MYWORKDIR # cd $MYWORKDIR
- prepare .rpmmacros
# echo "%_unpackaged_files_terminate_build 0" >> ~/.rpmmacros
- checkout the tagged code
# svn co svn+ssh://svn.cern.ch/reps/panda/panda-bigmon/tags/bigpandamon-core/bigpandamon-core-0.0.4
- cd to the tagged directory
# cd $MYWORKDIR/bigpandamon-core-0.0.4 # ls $MYWORKDIR/bigpandamon-core-0.0.4 core INSTALL.txt MANIFEST.in README.txt setup.cfg setup.py
- build RPM of bigpandamon-core-0.0.4
# QA_SKIP_BUILD_ROOT=1 python setup.py bdist_rpm ... lots of output, and then ... moving build/bdist.linux-x86_64/rpm/SRPMS/bigpandamon-core-0.0.4-1.src.rpm -> dist moving build/bdist.linux-x86_64/rpm/RPMS/noarch/bigpandamon-core-0.0.4-1.noarch.rpm -> dist # ls $MYWORKDIR/bigpandamon-core-0.0.4 core build dist INSTALL.txt MANIFEST MANIFEST.in README.txt setup.cfg setup.py # ls $MYWORKDIR/bigpandamon-core-0.0.4/dist -1 bigpandamon-core-0.0.4-1.noarch.rpm bigpandamon-core-0.0.4-1.src.rpm bigpandamon-core-0.0.4.tar.gz
- check files in the noarch RPM
- make sure that none of the python modules is missing (make sure every python module has init.py committed in SVN)
- make sure that none of the templates files is missing
- make sure that the static files are not missing
# rpm -qpl dist/bigpandamon-core-0.0.4-1.noarch.rpm | less
- remove .rpmmacros
# rm ~/.rpmmacros
Copy RPMs to the repository
- sudo su on aipanda004
- cd to the
distdirectory# MYWORKDIR="/data/jschovan/new-rpm/MYWORKDIR" # cd $MYWORKDIR/bigpandamon-core-0.0.4/dist
- cp src.rpm to /data/build/SRPMS
# cp bigpandamon-core-0.0.4-1.src.rpm /data/build/SRPMS/
- cp noarch.rpm to /data/build/x86_64
# cp bigpandamon-core-0.0.4-1.noarch.rpm /data/build/x86_64
Update RPM repos
- sudo su on aipanda004 (the pandabuild.cern.ch machine)
- cd to dir with repobuild scripts
# cd /data/build-not-public
- run the repobuild scripts
# ./repo_SRPMS # ./repo_x86_64
Deployment to the integration/production machines
Deployment to the integration machine
- sudo su on aipanda043 - integration
- to be continued...
Deployment to the production machine
- sudo su on aipanda022 - production
# yum clean all # yum upgrade
- make sure
local.pyis in place and is correct# /data/virtualenv/django1.6.1__python2.6.6__atlas/pythonpath/core/common/settings/local.py # /data/virtualenv/django1.6.1__python2.6.6__atlas/pythonpath/atlas/settings/local.py
- activate the
virtualenv- make sure "pip freeze" contains all the
virtualenvdependencies# cd /data/virtualenv/django1.6.1__python2.6.6__atlas # source bin/activate # export PYTHONPATH=/data/virtualenv/django1.6.1__python2.6.6__atlas/pythonpath:$PYTHONPATH
- make sure "pip freeze" contains all the
- try out
./manage.py runserver# cd /data/virtualenv/django1.6.1__python2.6.6__atlas/pythonpath/atlas # python manage.py runserver
- run
./manage.py collectstatic# python manage.py collectstatic
- restart the httpd service and change the owner of the logs files to apache:apache
# service httpd restart # chown apache:apache -R /data/atlpan/bigpandamon/var/log/* /data/bigpandamon_virtualhosts/atlas/logs/* /data/bigpandamon_virtualhosts/jedimon/logs/*
Major updates:
-- JaroslavaSchovancova - 24 Apr 2014
Responsible: JaroslavaSchovancova
Topic revision: r1 - 2014-04-24 - JaroslavaSchovancova
PanDA Web
Create New Topic
Index
Search
Changes
Notifications
Statistics
Preferences
- ABATBEA
- ACPP
- ADCgroup
- AEGIS
- AfricaMap
- AgileInfrastructure
- ALICE
- AliceEbyE
- AliceSPD
- AliceSSD
- AliceTOF
- AliFemto
- ALPHA
- ArdaGrid
- ASACUSA
- AthenaFCalTBAna
- Atlas
- AtlasLBNL
- AXIALPET
- CAE
- CALICE
- CDS
- CENF
- CERNSearch
- CLIC
- Cloud
- CloudServices
- CMS
- Controls
- CTA
- CvmFS
- DB
- DefaultWeb
- DESgroup
- DPHEP
- DM-LHC
- DSSGroup
- EGEE
- EgeePtf
- ELFms
- EMI
- ETICS
- FIOgroup
- FlukaTeam
- Frontier
- Gaudi
- GeneratorServices
- GuidesInfo
- HardwareLabs
- HCC
- HEPIX
- ILCBDSColl
- ILCTPC
- IMWG
- Inspire
- IPv6
- IT
- ItCommTeam
- ITCoord
- ITdeptTechForum
- ITDRP
- ITGT
- ITSDC
- LAr
- LCG
- LCGAAWorkbook
- Leade
- LHCAccess
- LHCAtHome
- LHCb
- LHCgas
- LHCONE
- LHCOPN
- LinuxSupport
- Main
- Medipix
- Messaging
- MPGD
- NA49
- NA61
- NA62
- NTOF
- Openlab
- PDBService
- Persistency
- PESgroup
- Plugins
- PSAccess
- PSBUpgrade
- R2Eproject
- RCTF
- RD42
- RFCond12
- RFLowLevel
- ROXIE
- Sandbox
- SocialActivities
- SPI
- SRMDev
- SSM
- Student
- SuperComputing
- Support
- SwfCatalogue
- TMVA
- TOTEM
- TWiki
- UNOSAT
- Virtualization
- VOBox
- WITCH
- XTCA
 |
|
Copyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback