Difference: LbSoftConfDb2 (1 vs. 2)

Revision 22019-03-15 - StefanGabrielChitic

Line: 1 to 1
 
META TOPICPARENT name="LHCbDevOps"

LbSoftConfDb2

Line: 6 to 6
 

LbSoftDb to LbSoftDb2 database migration procedure:

Added:
>
>
In order to migrate the LbSoftDb 1 database to the LbSoftDb2, you need to connect to any lxplus nodes and:
  1. Create a new virtualenv: virtualenv ./migrationEnv
  2. Activate the newly created virtualenv: source ./migrationEnv/bin/activate
  3. Install the migration library: pip install --extra-index-url https://cern.ch/lhcb-pypi/simple/ --trusted-host cern.ch lbsoftconfdbmigration
  4. Execute the migration: lb-sdb-migration Attention: The new database is dropped before the migration starts.
 

lbsoftdb.cern.ch new server VM initialization procedure:

LbSoftConfDb2 server installation steps (as root):

Line: 55 to 61
 . /build/env/bin/activate

lb-sdb-query --json listReleaseStacks

Changed:
<
<
EOT
>
>
EOT
 
  1. Create the cgi script for projects listing:
    cat <<EOT >> /var/www/cgi-bin/lb-projects 
    #!/bin/sh -x
Line: 68 to 73
 . /build/env/bin/activate

lb-sdb-dumpprojects -d

Changed:
<
<
EOT
>
>
EOT
 
  1. Assign execution permissions to the cgi-bin: chmod a+x /var/www/cgi-bin/*
  2. Verify that the apache proxy is working. Go in your browser to https://lbsoftdb.cern.ch/read/. If you get Service Unavailable means that the proxy is not working. The solution is to disable the SEL Linux: /usr/sbin/setsebool -P httpd_can_network_connect 1. If you get a Error response. Error code 501. means that the service is working.
  3. Congrats! You've made it wink

Revision 12019-03-15 - StefanGabrielChitic

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="LHCbDevOps"

LbSoftConfDb2

Service Maintenance and DevOps:

LbSoftDb to LbSoftDb2 database migration procedure:

lbsoftdb.cern.ch new server VM initialization procedure:

LbSoftConfDb2 server installation steps (as root):

  1. Start neo4j demon: systemctl start neo4j
  2. Install pip: easy_install pip
  3. Install virtual env: pip install virtualenv
  4. Create a virtual environment in /build/: virtualenv /build/env/
  5. Activate the newly created virtualenv: source /build/env/bin/activate
  6. Install LbSoftConfDb 2 server : pip install --extra-index-url https://cern.ch/lhcb-pypi/simple/ --trusted-host cern.ch lbsoftconfdb2server
  7. Install LbSoftConfDb 2 clients : pip install --extra-index-url https://cern.ch/lhcb-pypi/simple/ --trusted-host cern.ch lbsoftconfdb2clients
  8. Create the RO demon start script:
    cat <<EOT >> /build/startLBSoftConfDb2ReadOnlyServer.sh
    #!/bin/bash
    
    DIRECTORY=/build/
    export HOME=/root/
    source $DIRECTORY/env/bin/activate
    
    lb-sdb-server --listen=127.0.0.1:8080
    EOT
  9. Create the RW demon start script:
    cat <<EOT >> /build/startLBSoftConfDb2WriteServer.sh
    #!/bin/bash
    
    DIRECTORY=/build/
    export HOME=/root/
    source $DIRECTORY/env/bin/activate
    
    lb-sdb-server --writeAllowed --listen=127.0.0.1:8090
    EOT
  10. Login with neo4j/neo4j to http://lbsoftdb.cern.ch:7474 and change the neo4j password
  11. Create credentials in /root/private: mkdir -p /root/private & echo 'neo4j/new password > /root/private/neo4j.txt
  12. Restart the lbSdbServerReadOnly demon: systemctl restart lbSdbServerReadOnly
  13. Verify the restart was ok: systemctl status lbSdbServerReadOnly
    ...
    Listening for '127.0.0.1' on port '8080'...
  14. Restart the lbSdbServerWrite demon: systemctl restart lbSdbServerWrite
  15. Verify the restart was ok: systemctl status lbSdbServerWrite
    ...
    Listening for '127.0.0.1' on port '8090'...
  16. Create the cgi script for release stacks:
    cat <<EOT >> /var/www/cgi-bin/getreleases 
    #!/bin/sh -x
    
    echo "Content-type: text/json"
    echo ""
    export USER=apache
    
    . /build/env/bin/activate
    
    lb-sdb-query --json listReleaseStacks
    EOT
    
  17. Create the cgi script for projects listing:
    cat <<EOT >> /var/www/cgi-bin/lb-projects 
    #!/bin/sh -x
    
    echo "Content-type: text/json"
    echo ""
    export USER=apache
    
    . /build/env/bin/activate
    
    lb-sdb-dumpprojects -d
    EOT
    
  18. Assign execution permissions to the cgi-bin: chmod a+x /var/www/cgi-bin/*
  19. Verify that the apache proxy is working. Go in your browser to https://lbsoftdb.cern.ch/read/. If you get Service Unavailable means that the proxy is not working. The solution is to disable the SEL Linux: /usr/sbin/setsebool -P httpd_can_network_connect 1. If you get a Error response. Error code 501. means that the service is working.
  20. Congrats! You've made it wink

Architecture

LbSoftConfDb2 is web service that provides a Neo4j database used to store the dependencies of the LHCb software projects.

The project is composed of 2 packages:

  • a client side package that contains all the clients. The clients connect to the server side application using XML RPC via HTTP requests. The communication is done for clients that only query the databse using non-authe requests, while the database modifications are done using SSO auth requests.
  • a server side package. It provides 2 XML RPC servers: one for non-auth requests and the other one for auth-requests. The SSO auth is managed by a revers Apache proxy. The data exchange with the Neo4j server is executed only the main component of the application.
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback