Technical Road Map

Table of Contents


What is SIMPLE Grid Technical Road Map?

The SIMPLE Grid Technical Roadmap is a set of milestones for the core components and repositories of the SIMPLE Grid Framework that help meet the needs of site admins and end-users of the SIMPLE Grid Framework.

Milestones for upcoming releases of SIMPLE Grid Framework

SIMPLE Grid Framework 1.0 (Sep 2018)

Release Manager: Mayank Sharma

Overall Status:

Release Candidate Link to Kanban Board
Alpha https://github.com/orgs/WLCG-Lightweight-Sites/projects/1

Milestones

Feature Description Status Point of Contact Comments
The initial release of a SIMPLE Grid YAML Compiler Process variables, overriding of values, include statements etc. in the site level configuration file Ongoing

Mayank Sharma

Tarang Mahapatra

  • Links (ps, pointing to master branch):
  • Generates Augmented YAML files after processing variables, overrides and includes. More thorough testing required.
The initial release of a SIMPLE Grid Configuration Validation Engine Create validation rules for information present in site level configuration file Ongoing

Panos Paparrigopoulos

Mayank Sharma

  • Links (ps, pointing to master branch):
  • Basic architecture and support for basic validation rules completed
The initial release of a SIMPLE Grid Puppet Module The main central configuration manager for the first release of the SIMPLE Framework Ongoing Mayank Sharma
  • Links (ps, pointing to master branch):
  • 7 Aug: Waiting for Docker containers to be finalized
Use case demonstration with CREAM CE, Torque Batch System and Torque WN Demonstrate how the framework works from end to end in order to implement a Lightweight Site Ongoing Mayank Sharma

Supported Framework Components

The following component releases will be part of this release:

Supported Repositories

The following repositories will be part of this release:

Please check the Milestones for upcoming releases of Component Repositories section to go to their individual technical road map's and kanban boards for this release of the SIMPLE framework.

SIMPLE Grid Framework (Someday)

Feature Description Comments
Support for use of TOSCA templates as the Site Level Configuration File Analyse effectiveness of replacing Site Level Configuration file with more standardized TOSCA templates The DODAS framework developed by INFN uses TOSCA templates for describing infrastructure topology and orchestration information. Meeting with Daniele Spiga after CHEP 2018 made it a good candidate for incorporation in next major release of the SIMPLE framework.
Kubetnetes support for container orchestration Kubernetes provides a strong alternative to using Docker Swarm for container orchestration and is a widely popular tool. This feature aims to allow containers within the SIMPLE Grid Framework to be managed via Kunernetes. Julia Gavrilenko has started looking into the integration of Kubernetes. She has written Ansible playbooks to set up a Kube Cluster and deploy Torque WN containers. Here is the code on GitHub
Apache Mesos support for container orchestration Enable using Apache Mesos for container orchestration. An easy way to do so would be to integrate the DODAS. Mapping of our Site Level configuration file to a TOSCA template that can be read by the DODAS framework would be a good starting point to investigate.
Ansible as Central Configuration Manager Use Ansible Roles for configuration master node to function as the central configuration manager for the Framework Tarang Mahapatra started working on the Level-2 configurations using Ansible as part of Google Summer of Code 2018. Here is the code for the same: GitHub.
Detached MySQL MySQL server should not get destroyed when containers go down at a site.

Validate how reliable the existing MySQL containers are on DockerHub.

Provide a way for site admins to connect to an existing MySQL server on their site.

Support CVMFS and Squid Real workloads will require access to CVMFS and a local Squid proxy

CVMFS mount in containers was done by Maksim for his Alice VO-Box containerization project.

Squid containers are available on DockerHub.

Test APEL support Accounting is critical for Tier-2 sites. Protecting accounting data in a containerized environment needs to be tested/implemented Depends on the feature "Detached MySQL " as APEL data is stored in a MySQL database

Milestones for upcoming releases of Component Repositories

Milestones for existing Component Repositories

Component Name Upcoming Release Tied to SIMPLE Grid Framework Release? Links
Cream CE and Torque Batch System 1.16.4/2.5.13-1-origins Yes, v1.0 Technical Roadmap
Torque Worker Node 2.5.13-1-origins Yes, v1.0 Technical Roadmap

Support for New Component Repositories (Someday)

Feature Description Comments
Add support for ARC-CE New repository containing containerized ARC Compute Element Maiken Pederson from University of Oslo demonstrated containerized ARC-CE at CHEP 2018
Add support for HTCondor CE, Batch System, and Worker Node Assess the challenges involved with containerizing HTCondor and break them down into appropriate repositories

Personal Condor : https://github.com/maany/personal_condor_docker

HTCondor Pool : https://github.com/maany/htcondor-docker-pool

-- MayankSharma - 2018-08-06
Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r5 - 2018-09-27 - EraldoSilvaJunior
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback