---

Developer's Guide

Location of the files:

• SVN: https://svn.cern.ch/reps/totem/trunk/runlog/
• AFS: /afs/cern.ch/user/t/totemdbk/
• (current) working version can be found at: https://svn.cern.ch/reps/totem/tags/runlog/runlog_production

Important: the application is based on Django framework: http://www.djangoproject.com/. Everything is made in accordance with Django convention. Before any large modifications in the source code, you should get to know Django (docs).

Directory at SVN contains two directories:

• dataset
• www

how (and where) to make changes

In AFS directory: /afs/cern.ch/user/t/totemdbk/working_directory/runlog there is a version of the TOTEM Runlog system prepared for changes and testing.

You can make changes in the project in that directory (below there are sections which provide more information how to do it). Later, test it by running a development server:

cd /afs/cern.ch/user/t/totemdbk/working_directory/runlog/dataset

./manage.py runserver 0.0.0.0:12345

This command runs the development server on port 12345. To check IP of the lxplus machine execute:

./ip.sh (located in cd /afs/cern.ch/user/t/totemdbk/working_directory/)

Now, you can test website by putting address in your webbrowser (let's assume that IP is: 137.138.4.21): http://137.138.4.21:12345/runs

If everything works fine, copy changed files from /afs/cern.ch/user/t/totemdbk/working_directory/runlog/dataset/[...] to cd /afs/cern.ch/user/t/totemdbk/dataset/[...] and take a look at deployment section.

Be careful with settings.py file! Its version in working_directory is completely different from the version in the production directory. So, do not overwrite settings.py file!

Of course you can do changes locally on your machine. You have to install: Python 2.4 or later, Django 1.2.1, cx_Oracle library for database support.

deployment

The webserver has to support FCGI and python (>2.4). Currently there is the webafs01 server, which supports these technologies.

Site managment: https://webservices.web.cern.ch/webservices/Services/ManageSite/Default.aspx?SiteName=totem-runlog (login as totemdbk)

To put the new version into production:

1. Copy directories dataset and www into /afs/cern.ch/user/t/totemdbk/www/dataset and /afs/cern.ch/user/t/totemdbk/www (or just modified files)
2. To make changes visible (restart FCGI process):
   1. login as: totemdbk at: https://webservices.web.cern.ch/webservices/Tools/SiteConfiguration/?SiteName=totem-runlog&Folder=fcgi
   2. Uncheck field Enable CGI execution, click Update
   3. Wait a few minutes until website is dead (403 Forbidden error should be visible)
   4. Check field Enable CGI execution, click: Update
   5. That's all, changes should be visible now

To put new version into another site/account:

1. The Account on which you want to deploy an application should be the service account as the application tries to write logs to the server log directory
2. The code of application shoul be located in directory which parent is www directory
3. When you checkout code you should ensure that content of the sites is available for the reading. In case you deploy application on the afs some useful information can be found at: https://espace.cern.ch/webservices-help/Websitemanagement/ConfiguringAFSSites/Pages/PermissionsforyourAFSfolder.aspx
4. Fill application credential for the database connection. In file settings.py should be blank spaces.
5. On the cern webservices web page create site account and configure it properly(enable cgi scripts on the runlog/fcgi directory).

Step II is not necessary if you change only static files like HTML templates, CSS styles etc.

GUIDE

Part I: dataset directory, making changes in the project, database structure

dataset directory

Django project with the TOTEM Runlog application.

Should be put into: /afs/cern.ch/user/t/totemdbk/www/dataset

The most important files:

dataset/settings.py - settins of the project

You have to input manually:

• database username and password
• settings of email account used for sending opinions (optional)

You can easily change:

• date format (option DATE_FORMAT)
• default fields which are presented on the list (option LIST_FIELDS)
• fields which are disabled for test beam (option TEST_BEAM_DISABLED_FIELDS)
• number of runs per page (option PAGINATE_BY)

dataset/runs/models.py - declarations of data structures

dataset/runs/views.py - application logic

dataset/runs/templates/* - presentation layer, HTML templates, easy to modify

dataset/runs/media/ - directory which contains media files (CSS, images, JavaScript)

Special, nonstandard files prepared for easy modification:

dataset/runs/units.py - list of units (used in new run form, run details, elog entry)

dataset/runs/help.py - help content for fields in new run form

HOWTO

I want to change / disable the unit for the parameter X:

• to change the unit just enter the new value in the dictionary UNITS in units.py
• to disable the unit for some field, use value: u'' (i.e. u'energy': u'' causes that unit for energy is not displayed)

I want to change / disable the help text for the field X:

• to change the help text just enter the new value in the dictionary HELP_TEXT in help.py
• to disable the help icon for some field, use value: None (i.e. u'run_number': None causes that help icon and help text for 'run number' field is not displayed)

I want to change the name of the parameter X:

• add / modify attribute verbose_name=u'New name' of the field X in class Run / RomanPotConfig / T1Config / T2Config in models.py

Below there are more complicated actions, which require knowledge of SQL and Django / python. You will need to do some changes in the database - its description is below the HOWTO section. Following actions can result in lose of the data!

I want to add a new parameter describing a run:

1. add a new field in an appropriate model in dataset/runs/models.py
2. modify templates/run/new_run.html (look how other fields are rendered and do it analogically for a new field)
3. modify templates/run/rundescription_detail.html or templates/run/config_detail.html (look how other fields are rendered and do it analogically for a new field)
4. modify to_plain_text function in the model which contains a new field - used for saving data in the elog system (look how other fields are converted and do it analogically for a new field)
5. modify units.py and help.py if you need units / help text
6. apply a SQL script which will add a new column in the table. Check how Django maps your field type into a database column (in documentation or look at existing fields and corresponding columns). I.e. in Oracle database: TextField -> NCLOB, CharField -> NVARCHAR2, IntegerField -> NUMBER(11,0), FloatField -> FLOAT, DateTimeField -> TIMESTAMP(6). Important: remember to add the constraints (unique, not null) in SQL script corresponding to parameters of your new field.

I want to change the type of a parameter (i.e. from integer to string):

1. modify the field in the appropriate model in dataset/runs/models.py
2. apply a SQL script which will change datatype of the column in the table. Check how Django maps your field type into a database column (in documentation or look at existing fields and corresponding columns). I.e. in Oracle database: TextField -> NCLOB, CharField -> NVARCHAR2, IntegerField -> NUMBER(11,0), FloatField -> FLOAT, DateTimeField -> TIMESTAMP(6). Important: remember to add constraints (unique, not null) in SQL script corresponding to parameters of your new field.

I want to make one of the parameters required:

1. modify the field in the appropriate model in dataset/runs/models.py - remove null=True, blank=True params
2. apply a SQL script which will remove a NOT NULL constraint from the appropriate column in the table.

I want to make one of the parameters optional:

1. modify the field in the appropriate model in dataset/runs/models.py - add null=True, blank=True params (only blank=True for! CharField or TextField!)
2. apply a SQL script which will add NOT NULL constraint to the appropriate column in the table.

I want to remove one of the parameters:

Just don't do it. It's better and safer to make it optional.

Database

CERN Oracle Production Database:

APPLICATION ACCOUNT:

• username: TOTEM_RUNLOG_2012
• connection string(db alias) : totem_prod
• host : pdbr
  

WRITTER ACCOUNT:

• user name : TOTEM_RUNLOG_2012_W
• connection string(db alias) : totem_prod
• host : pdbr

READER ACCOUNT:

• user name : TOTEM_RUNLOG_2012_R
• connection string(db alias): totem_prod
• host : pdbr

The direct connection could be created thanks to the above information and cern official tnsnames.ora file.

The diagram shows most important tables in the database (prefix RUNS) - mappings from Django models into Oracle tables. You can see how Django fields are changed into table columns.

Another tables in the database:

• with prefix DJANGO - Django specific tables, do not modify (they are described in Django docs)
• with prefix AUTH - Auth module, enabled because Admin Panel needs it

Part II: www directory, deployment, maintenance

www directory

Contains settings necessary for the production environment.

Should be put into: /afs/cern.ch/user/t/totemdbk/www/runlog

Content:

runlog/www/
    admin_media
    .htaccess
    fcgi/
        dataset.fcgi
        .htaccess
    media
    readme.txt

• runlog/.htaccess - Apache settings for redirection (/totem-runlog -> /totem-runlog/fcgi/runs)
• runlog/fcgi/dataset.fcgi - FastCGI script for Django project
• runlog/fcgi/.htaccess - Apache settings for authentication (contains group with access!)

The directory should contain two additional symbolic links:

• to ../dataset/runs/media called media
• to ../lib/Django/django/contrib/admin/media/ called admin_media

If you are logged as totemdbk and you are in /afs/cern.ch/user/t/totemdbk/www/runlog directory, you can create them by typing:

ln -s ../dataset/runs/media media

ln -s ../lib/Django/django/contrib/admin/media admin_media

libraries modification

Run Log application use 2 different accounts in order to communicate with the database(readinf, writting accounts), but database space belongs to the database application account. In orther to perform database operations using mentioned reading writting accounts there is need to change current schema for this 2 accounts. In Django 1.3 version do not provide a such mechanism so Run Log use slightly modified Django libraries. In fact performed changes enable executing initializing queries like setting current schema. All changes all described below:

• add import to the file django/db/backends/oracle/base.py (instead of oracle there may be some other database management system like postgres etc.)
  from django.conf import settings
• find line contaning this piece of code(it will be at more or less in 370 line number)
  cursor.execute("ALTER SESSION SET NLS_DATE_FORMAT = 'YYYY-MM-DD HH24:MI:SS'"
  "NLS_TIMESTAMP_FORMAT = 'YYYY-MM-DD HH24:MI:SS.FF' "
  "NLS_TERRITORY = 'AMERICA'")
  above line you found paste this code:
  for init_query in settings.CONNECTION_INIT_SQL: cursor.execute(init_query)
• in the file django/conf/global_settings.py add line:
  CONNECTION_INIT_SQL = (
  #'ALTER SESSION SET CURRENT_SCHEMA = SCHEMA',
  # changing conenction schema in oracle database to the SCHEMA
  )

maintenance

If you want (or have to) change Django version just put the new version into /afs/cern.ch/user/t/totemdbk/www/lib/Django/ (the directory should contain the subdirectory django with the library itself - take a look how it is made now).

If you want to use specified Python version change first line od FCGI script dataset.fcgi from #!/usr/bin/env python to #!/usr/bin/env pythonXY, i.e. #!/usr/bin/env python26. Currently Python 2.6 does not work at the webafs01 server (I don't know why - Juraj Sucik is responsible for that server).

/afs/cern.ch/user/t/totemdbk/app - directory contains applications needed for the Elog support (elog client application). Probably there will be no need to touch it. If you change something, remember to change path to elog client app in /dataset/settings.py file.