Adding External Components

When a user requests and externals, an email is sent to etics-support.

On the ETICS Portal

Following the link in the email gets you to the portal page where externals are managed.

The portal will prompts you 4 pages.

• Summary

Summary information. The reason can be used to decide the priority of the request. If the request is for a new configuration of an existing component, the component will have an id. Existing configurations can be used as model for the new.

• Component Details

Check the information make sense. Download the provide binary and check is in the correct format for the repository.

• Configuration Details

Here goes the configuration information. Check the version and the platform. If the binary is platform independent, change ${platformName} to noarch in the repository path.

For components that are platform independent, change ${platformName} to noarch in the repository path text box.

• Binary Location

Check the binary location. Then accept or reject the request, giving exhaustive info when rejecting. If the location has not been set contact the user who made the request or find the binary by yourself remembering that the file has to be in tar.gz format and not zip or other format, in this second case unzip and tar it again.

The last page generate the shell commands you will run inside the CERN afs (more or less).
Don't close the page, you'll hit the next button once done the next step.

On lxplus.cern.ch

Installing the repository client

The first time you do that you'll have to install the repository client application, available at http://etics-repository.cern.ch:8080/repository/download/registered/org.etics/org.etics.repository.client-java/1.3.2/noarch/etics-repository-client-java-1.3.2-1.tar.gz

Untar to a directory of your choice, then edit the file etc/eticsRepositoryClientJavaConfiguration.xml providing your keystore, certificate (in p12 format, if not convert it with openssl) and password. The keystore is available as attachment.

Setting up the Configuration XML file

• Obtain a suitable JKS file to be defined in the etc/eticsRepositoryClientJavaConfiguration.xml

  wget https://ca.cern.ch/ca/CRL/CERN%20Root%20CA.crt -O cern.crt
  keytool -import -alias cernca -file cern.crt -keystore truststore.jks
  

• Create a pkcs12 version of your certificate. To convert from PEM to P12:

  openssl pkcs12 -export -in cert.pem -out cert.p12
  

• Note: if your cert.pem does not contain both the public and private keys, you can not use it for this purpose. Instead, your CA may have generated a p12 for you by default (use this instead).

Adding an external

Depending on how you installed the client, you may need to modify the generated commands. Let's see them in details.

• mkdir -p /afs/cern.ch/project/etics/repository/externals/${component-name}/${version}/${platform}/

This creates the directory in the repository where the artifacts will be stored.

• cd /afs/cern.ch/project/etics/repository/externals/${component-name}/${version}/${platform}/

This cd to the previously created dir.

• wget -v -O ${component-name}.tar.gz http://url-path/{component-name}.tar.gz

This downloads the artifacts.

If you haven't specified an age, the generated commands will contain an unnecessary - after the version. Remove it.

• * java -jar /opt/etics/lib/eticsRepositoryClient.jar register-file -l externals/${component-name}/${version}/${platform}/${component-name}.tar.gz -t tar.gz ${component-name}.tar.gz*

EXAMPLE:

• java -jar /opt/etics/lib/eticsRepositoryClient.jar register-file -l externals/commons-lang/2.4.0/noarch//commons-lang-2.4.0-.tar.gz -t tar.gz commons-lang-2.4.0-.tar.gz

You can run this command as it is only if you have the repository client jar installed under /opt/etics. If not, point the -jar option to your installation directory, and use the configFilePath property to point the configuration file. The following example assumes the installtion directory is $HOME/opt/etics-repository-client-java-1.0.4-1.

java -DconfigFilePath=/afs/cern.ch/user/v/vventuri/opt/etics-repository-client-java-1.0.4-1/etc/eticsRepositoryClientJavaConfiguration.xml -jar \ /afs/cern.ch/user/v/vventuri/opt/etics-repository-client-java-1.0.4-1/lib/eticsRepositoryClient.jar \
register-file -l externals/commons-lang/2.4.0/noarch//commons-lang-2.4.0-0.tar.gz -t tar.gz commons-lang-2.4.0-0.tar.gz

Remove the double // from the generated command before running them.

NOTES:

• Upload to /afs/... is possible from etics-repository server.
• The client to register files is available in ETICS server.
• Ask to the user that demanded the file if it is working correctly for him before to lock the new external.
• Since lxplus has changed over to using SL 5.x there may be a requirement to set:

  export JAVA_HOME=/usr/lib/jvm/java-1.6.0-sun-1.6.0.20.x86_64/jre
  export PATH=$JAVA_HOME/bin:$PATH
  

  to avoid the use of openjdk and instead use SUN jdk.

Removing an external

A similar command but with "-r" option instead.

* * java -jar /opt/etics/lib/eticsRepositoryClient.jar remove-file -r externals/${component-name}/${version}/${platform}/${component-name}.tar.gz*

Use of Alien conversion tool

To convert a file with an extension to another kind of file use the Alien conversion tool

Alien is a tool to convert the installation packages. For example if you need to convert from .rpm to .tgz you can use the following command:

sudo alien --to-tgz fileName-version.rpm

This command converts the package to fileName-version.tgz In the same way you can convert the package from one of the most common format, Red Hat (.rpm), Debian (.deb), Stampede (.slp), Slackware (.tgz), and .tar.gz, to an other of them.

Note that if your package is named like iSeriesAccess-5.4.0-1.4.i386.rpm the "sudo alien -d -k iSeriesAccess-5.4.0-1.4.i386.rpm " converts it to debian version but introduce a difference in the name, that will be "iseriesaccess-5.4.0-2.4_i386". If you don't like this behaviour you should use "-k" option, so the command converts the .rpm to deb, without changes in the naming and versioning.

Create external configuration

Build time dependencies (.tar.gz.)

STEPS:

* Find the tar.gz

* Extract it and put $ModuleName.location to XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx

* Register in both repositories (copy the file and use the repositoryClient)

* Make sure "downloadFrom" match (click link)

* Add DEFAULT platform (property "package.buildArch=noarch")

* Check tarball structure. It must be "moduleName-Version", without age, and inside the different directories (bin, src, etc).

Run time dependencies (.deb or .rpm)

* If A has a dependency in B version x.y.z, then package A requires B >= x.y.z

* rpm/deb content are metadata + files. The metadata contains: package name, version (x.y.z-age), provides (tokens with package names), requires (list of package names or package names with >=), obsoletes.

* In ETICS there are 3 properties:

• package.versiondeps

• package.versionwithAge

• package.strictdeps

• Possible cases depend on their content can be:

• • If all of them empty, then B is required.

• • if package.versiondeps is true, then dependency is >=

• • if package.versionwithAge is true, then it takes also the age

• • if package.strictdeps is true, then dependency is =

* There is two names, packageName and moduleName. If packageName is empty, then moduleName is used.

* If the rpm name does not match we can do 2 things:

* Add in the property package.rpm.name the rpm name (Do not do this for the default platform).

* If it is already locked, then create a symbolic link (ln -s ...)

NOTE:

* Ask for each external if it is runtime dependency or it is not.

* RPM name structure: packageName-VersionWithAge.Platform.Architecture.rpm or packageName-VersionWithAge.noarch.rpm

Create rpm from tar to be added.

* Ask to the user:

* Exact version: x.y.z-age

* Package name

* Package prefix (where it is going to be installed. Ex: /usr/lib, /opt/etics, etc..)

* Requires: list of "module >= version"

* Add in check-out command:

* mkdir -p ${moduleName}

* Add in build command:

* init = wget tarball

* compile = tar -xzf tarball

* install = mv lib ${targetDir} (same with bin, etc...)

* Check that the module name provided by the user match our modules or package name.

* Property: package.prefix = Where you want to have installed the package. Ex. /usr/local/java.

* Property package requires = packageName >= x1.y1.z1, packageName >= x2.y2.z2

* Then, build and register

If user ask to register a .jar

* Create a tar.gz file with inside: /component-version/lib/component.jar

• Create tar : tar -pczf x.tar.gz file

• Open tar: tar -xzvf x.tar.gz

Finally

Make sure to check

• with the web application that the conf get created.

• send a build with "check out run-time dependencies" to check the system is able to find the file.

The hit next in the portal to send notification of approval to the user.