Results of the Interviews of ETICS Users

Introduction

Interviews to ETICS users involved 9 people on the whole.
  • 3 from CNAF
    • Andrea Ceccanti - developer, integrator of org.glite.voms
    • Vincenzo Ciaschini - developer, integrator of org.glite.voms
    • Fabio Capannini - integrator of org.glite.wms
  • 4 from CERN
    • Paolo Tedesco - developer of org.glite.data.fts
    • Joni Hahkala - developer and integrator of org.glite.security
    • Andreas Unterkircher - integrator of org.glite
    • Francesco Giacomini - integrator of org.glite.wms
  • 2 from INFN-Padova
    • Alvise Dorigo - developer of org.glite.ce
    • Paolo Andreetto - integrator of org.glite.ce

Below there are the answers to the most important questions asked to them.

Evaluation of performance and usability about ETICS services

Performing which task do you think that ETICS is useful to you?

  • Managing of the dependences (4 people)
  • Possibility to provide to our clients packages that they can install in a certain platform without any problems (for example I can create an rpm with all that if useful to install it - metapackage) (3 people)
  • Remote build on different platforms (3 people)
  • Creation of workspace (in every moment), and build without any problems. (3 people)
  • Good reports for builds
  • Possibility to have repeatable build

In what aspects, instead, do you think that etics delays your activity?

  • etics-checkout was slow with etics client 1.3 but now with etics-client 1.4 is ok (3 people)
  • Slow checkout with etics command line client
  • Editing of the configuration too much slow in the WA, WA interface too much slow,
  • Command line client non reliable, when I do some commit the dependencies got lost, or when I clone one configuration some dependencies disappear in the new configuration. Sometimes this happens also when I do etics-configuration prepare
  • etics-build slow. Usually I extract some instructions from log.xml file and insert them in a script that I execute.
  • etics-build slow. etics-build execution makes the configure and the rpms every time. I would like that etics-build execution stops at stage target and that it would not make configure if not necessary
  • remote build slow
  • etics CLI difficult to install in some platforms like UBUNTU
  • etics-tag command does not manage the properties' values

Concerning etics command line client:

Which commands do you use more frequently?
  • etics-checkout ++++ (5 people)
  • etics-build ++++ (5 people)
  • etics-submit
  • etics-configuration

What about their performance and usability?
  • etics-checkout performance really improved since version 1.4 (3 people)
  • some commands with some options are difficult to understand (for example --merge, --replace) (2 people)
  • options of etics-checkout, etics-build are difficult to understand
  • ok concerning usability after having read the user manual (keep the manual updated!)
  • etics-configuration: performance ok, problems concerning usability.
  • not very clear if for some commands there is a network connection with the server
  • not very clear what metadata contain
  • semantics of certain operations is not very clear
  • documentation unsatisfactory (not complete or not updated)
  • documentation too much summarized, how can I know all the implications of a certain command with a certain property?

Do you use etics command line client only for integration or also for developing?*
  • I use ETICS only for integration tasks and not in my daily activity (5 people)
  • I use ETICS in my daily activity (3 people)

Concerning etics web application:

Do you think that the placement of the various commands (buttons) is intuitive?
  • Not very much, but it is not important (3 people)
  • Yes (4 people)
  • I do not use the buttons in the configuration panel... I prefer contextual menus
  • Insert some description about what are the functions of each button to make them more clear

Do you think that a different placement consisting in grouping the buttons in different panes will be more intuitive?
  • No (5 people)
  • Yes, but it is not very important
  • No at all, it is space consuming

Do you ever felt the need to have a guide that leads you in using the proper button in some situations?
  • No (4 people)
  • Yes but a wiki guide external to the web application with some examples (3 people)
  • Yes but a pdf documentation with some examples

Do you think that it would be useful to have a sort of guidelines that explains to you how to fill the form correctly? For example dependencies, properties, or configurations...
  • No (6 people)
  • No, a good documentation is enough

Which commands (or buttons) do you use more frequently in the web application?
  • editing of configurations (7 people)
  • clone configurations (6 people)
  • locking of configurations (6 people)
  • remote build (5 people)

What do you think about the overall layout? Where do you think it can be improved?
  • customizable view focusing on the projects on which I am working (2 people)
  • satisfactory, I would like to have 2 configuration or 2 subsystem in the same window. With 2 subsystem I can deduce what components are different (2 people)
  • good. I would like an updated documentation
  • no intuitive at first time

What panels are lacking in usability?
  • greater coherence between what is shown in the right panel and what is in the workspace.(2 people)
  • right click on configurations, the menu is shown too much slowly
  • submit a remote build is too much slow
  • show only the machine that are actually available when I am doing a remote build
  • it should be mandatory to work only in configurations within a workspace

What are the actions you find difficult to accomplish (e.g. too many clicks, unclear sequence, insufficient feedback, etc..)?
  • visualization of a configurations (too many clicks) (2 people)
  • it is difficult to understand how I can add a dependency (2 people)

Where do you find the WA is not presenting concepts coherently with the CLI?
  • no, I use WA and CLI for different purposes (3 people)

Is there any functionality you feel the WA misses at interface level?
  • no
  • diff between configurations
  • reverse dependency resolution
  • no. they are equivalent (free text in the web app it is difficult to understand what and where insert the information)

Is there any piece of the WA you feel useless?
  • no (5 people)
  • the button on top panel in configuration tab: they are redundant

Other information about evaluation

  • Reports with a structure not much comprehensible (At the beginning stdin and stderr at the end only stderr)

List of requirements

ETICS Command line client

  • Some commands do not require a workspace setup (i.e. etics-submit), and it should be possible to execute them without creating one (2 people)
  • at the moment etics-checkout makes a call for each dependency. A good improvement could be that etics CLI makes one check about the dependencies missing and makes the download of all of them
  • possibility to delete a remote build
  • more machines SL4 with 32 bit
  • making the autocomplete with TAB such as bash
  • refactoring of ETICS commands in a easier way
  • Better explanation about what etics commands (for example etics-tag) do
  • About interactive commands better explanation of their options
  • etics command line should ask for password at the beginning of the execution of a command (etics-checkout or etics-build)
  • possibility to build from the same workspace in more than one platform (for example slc4 AND slc5). Shared workspace and then build with different virtual machine
  • possibility to run etics-build without specifying any component. It builds the component of the latest checkout
  • Add a command like etics-show-tag to know what tag I have in my workarea
  • ETICS CLI should be able to resolve all the dependencies with the information stored in the local workspace, without connecting every time with the web service
  • etics-build should avoid ./configure if it has been already done
  • automatic analysis about conflicts among configurations with warning
  • etics-build execution stops at target "stage"

ETICS WA

  • property drop-down list when typing ${ with description of them (4 people)
  • Multiple configurations in a workspace (3 people)
  • Rethink "add dependency" more intuitive or explain it in the dependency section after the dependencies (3 people)
  • Notification system in order to know who modified a certain configuration or who created a new configuration (2 people)
  • Given a certain component I would like to know who can do what action for that component? (2 people)
  • Cookies in order to save the state of the web applications (2 people)
  • session Expiration longer
  • provide a way to test the commands inserted without having to submit a build and wait 30 minutes to find out that you mispelled a command
  • when missing permission, it should show a panel where to ask for it, the request goes to the module administrators
  • link the role information in the add permission page
  • Reg expression to be used in filter of configuration
  • Add multiple entries in workspace: configurations (or subsystem) comparison
  • Divide the Properties section by the property category such as Dynamic dependency definition, Custom properties, Packaging properties, Build process properties, maybe also showing what are the properties that are generated from the configuration, component (such as age, name, majorVersion), etc module.DEFAULT at the moment is a free form, a drop-down would be more useful starting with the most recent (explain better)
  • possibility to delete a remote build
  • Author/maintainer missing
  • Not very clear what are the permissions belonging to a certain role. For example which role you should have to lock (or clone) a configuration?
  • When cloning a configurations cloning also of the permissions on that configurations
  • Too many roles. Necessary updating of roles description providing meaningful examples
  • Possibility to execute the build of a certain component choosing the components with which to build it (Horizontal Build)
  • In the project panel of the tab configuration of the WA I would like to see only the projects (subsystems or components) which I am working on (or to have a customizable view of the list of project shown in the project panel)
  • When I click on a configuration in the configurations panel I would like to see the details of this configuration (subsystems' configurations, or VCS, build commands without having to insert the configuration into the workspace)
  • In case of dynamic dependency, when I set default for a component I would like that the WA showed what is the version that will be take into consideration (according to a certain project-config) even if the configuration is not locked
  • Description that could show me information about a particular branch
  • Clone command I would like that it update automatically the version of the new configuration (for example from R_2_0_16-1 to R_2_0_17-1)
  • Improve administration tab with new functionalities
  • Possibility to do the editing of the properties in a pop-up window

Repository

  • yum repository (3 people)
  • bookmark to see where my rpm were generated
  • look at report of a certain build without knowing when it was built
  • file search in packages in the repository
  • redundant repository

Other Requirements

  • monitoring more responsive about the situation of the builds (better estimate of the remaining time to complete a build)
  • analysis about conflicts among configurations made by ETICS
  • have the possibility to build components using more than one core

Conclusions

Evaluation of performance and usability

ETICS Command line client

  • Most of interviewed people considers the performance of etics-checkout command really improved with version 1.4.
  • There are some complaints concerning etics-build (and remote build) performance. Most of the users, in order to avoid to waste too much time, prefer, if possible, to use autotools, or ant commands.
  • Some users said that concerning some options, like --merge, or --replace, is difficult to understand precisely what they do.
  • Most of interviewed admit that the documentation (ETICS User Manual ) is very useful to understand commands and options of ETICS CLI, but they would like that this guide is more frequently updated
  • In order to avoid to spent to much time with ETICS most of developers I interviewed said that they do not use ETICS in their daily activity. But they prefer to use it when it is necessary to do a new release of the software

ETICS Web Application

Most of the people I interviewed consider the WA a good tool for editing and modifying and locking of configurations and making remote build.
  • Some users complaint about general performance of etics WA.
  • Some of them said that It is difficult to add a dependency in a configuration

Requirements

ETICS Command line client

Analyzing user's suggestions about how to improve ETICS CLI, it is very difficult to assign a priority to them. There is no requirement that gathered many more votes respect to others. In order to do a classification it is possible to say that, in order to improve performance
  • people interviewed would like to have an etics-submit that does not depend on etics-workspace-setup command.
  • moreover they underlined that it is annoying that ETICS CLI, during a checkout, resolves dependencies one by one, in their opinion this is very time consuming.
instead to improve usability:
  • users suggested to explain better ETICS command and options,
  • etics-checkout should ask for passwords at the beginning of the execution. In this way it would be possible to do some other activities during checkout.
  • It should be possible to execute etics-build without any arguments and it should build by default the component that has been checked out last time.
  • etics-show-tag that should show the tag of the code in a certain workarea

ETICS Web Application

Concerning web application a lot of requirements were gathered all of them are about the configuration tab. The most important are:
  1. property drop-down list when typing ${ with description of them (4 preferences)
  2. Multiple configurations in a workspace (3 preferences)
  3. Rethink "add dependency" more intuitive or explain it in the dependency section after the dependencies (3 preferences)
  4. Notification system in order to know who modified a certain configuration or who created a new configuration (2 preferences)
  5. Given a certain component I would like to know who can do what action for that component? (2 preferences)
  6. Cookies in order to save the state of the configuration tab in web applications. (2 preferences)

Repository

  • Concerning repository the most important request is about yum repository

-- MarcoCanaparo - 09 Dec 2008

Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r8 - 2009-04-03 - MarcoCanaparo
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    ETICS All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback