Green Big Data Project

The Green Big Data is a collaboration project between Aalto University and Helsinki Institute of Physics. The project develops methods for minimising energy consumption and maximising performance of computing resources. The current project started 1st of January 2014 after the previous 3-year GreenIT project.

The motivation behind the project comes from the LHC experiment of CERN producing about 15 petabytes of data in a year. In this scale, even a small optimisation can offer noticeably energy and cost savings. Environmental awareness and high-energy prices have made energy efficiency a new important issue in IT systems, since energy consumption has clearly become one of the main costs of computing. Based on this it has been estimated that the electricity cost can easily exceed the hardware cost during the lifetime of the system. In computing intensive sciences, such as high-energy physics (HEP), energy efficient solutions are important but little research effort has focused on the area. The main collaboration partners of the project are CERN IT and CERN experiments and the Department of Operations at the University of Lausanne.

Related Projects

The Scalable and Secure Infrastructures for Cloud Operations (SSICLOPS) project focuses on techniques for the management of federated private cloud infrastructures, in particular cloud networking techniques within software-defined data centres and across wide-area networks. SSICLOPS is funded by the European Commission under the Horizon2020 programme. https://ssiclops.eu/

Selected publications

  • Niemi, T., Nurminen, J.K., Liukkonen, J.-M., and Hameri, A.-P., “Towards Green Big Data at CERN,” Future Generation Computer Systems (to appear) pdf
  • Kimmerlin, M., Hasselmeyer, P., Heikkilä, S., Plauth, M., Parol, P., Sarolahti, P, "Network Expansion in OpenStack Cloud Federations", European Conference on Networks and Communications (EuCNC), Oulu, Finland, June, 2017 pdf
  • Khan, K.N., Scepanovic, S., Niemi, T., Nurminen, J.K., Von Alfthan, S., and Lehto, O-P, “Analyzing the Power Consumption Behavior of a Large Scale Data Center”, 2nd Workshop on Energy-Aware High Performance Computing (EnA-HPC), Frankfurt, Germany, June 2017
  • Kimmerlin, M., Plauth, M., Heikkilä, S. and Niemi, T., " A Practical Evaluation of a Network Expansion Mechanism in an OpenStack Cloud Federation", CLOUDNET'17, forthcoming. pdf
  • Nurminen, J.,K., Niemi, T., Strandman, J., Ruokosuo, K., "Sunburn - Using Excess Energy of Small Scale Production for Distributed Computing", Energy Efficiency, in press. pdf
  • Abaunza, F., Hameri, A-P., and Niemi, T., "EEUI: A new measure to monitor and manage energy efficiency in data centers", International Journal of Productivity and Performance Management, in press. pdf
  • Kommeri, J, Niemi, T., and Nurminen J.K, “Energy efficiency of dynamic management of virtual cluster with heterogeneous hardware” The Journal of Supercomputing, 73(5):1978-2000, 2017 pdf
  • Khan, K. N., Hoque, M. A., Niemi, T. P., Ou, Z. & Nurminen, J. K., "Energy efficiency of large scale graph processing platforms", Ubicomp2016, Heidelberg, Germany, Sep 2016 p. 1287--1294, 2016.
  • Khan, K.,, Ou, Z., Hirki, M., Nurminen, J.K., and Niemi, T., "How Much Power Does your Server Consume? Estimating Wall Socket Power Using RAPL Measurements," ISC workshop on Energy-Aware High Performance Computing, June 2016
  • Kommeri, J., Heikkilä, S., Vartiainen, A., and Niemi, T., “The Effect of Networking Performance on High Energy Physics Computing” International Conference On Green IT Solutions, June 2016 pdf
  • Nodari, A., Nurminen, J.K., and Frühwirth, C., “Inventory Theory Applied to Cost Optimization in Cloud Computing,” 31st ACM/SIGAPP Symposium on Applied Computing (SAC 2016), Pisa, Italy, 2016 pdf
  • Hirki, M., Ou, Z., Khan, K.N., Nurminen, J.K., Niemi, T., “Empirical Study of the Power Consumption of the x86-64 Instruction Decoder,” Usenix Workshop on Cool Topics in Sustainable Data Centers (CoolDC'16), Santa Clara, CA, USA, March 2016 pdf
  • K. Nizam Khan, F. Nybäck, Z. Ou, J. K. Nurminen, T. Niemi, G. Eulisse, P. Elmer and D. Abdurachmanov. Energy Profiling using IgProf. CCGrid 2015, 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, May 4-7, 2015, Shenzhen, Guangdong, China. (A poster presentation) pdf
  • J. K. Nurminen, J. Strandman, K. Koskela and T. Niemi. Computing Heaters - An Energy-efficient Way to Provide Computing Services. ExtremeGreen 2015: Extreme Green & Energy Efficiency in Large Scale Distributed Systems, May 4-7, 2015, Shenzen, Guangdong, China pdf
  • M. A. Hoque, M. Siekkinen, K. N. Khan, Y. Xiao, S. Tarkoma. Modeling, Profiling, and Debugging the Energy Consumption of Mobile Devices. ACM Computing Surveys, October 2015. pdf
  • F. Abaunza, V. Chavez-Demoulin, A-P. Hameri, and T. Niemi. Do flow principles of operations management apply to computing centres? Production Planning & Control, 26(4), 2015. pdf
  • J. Kommeri, F. Abaunza, M. Niinimäki, T. Niemi, and P. Thanisch (2014) Energy efficiency of a web and database server system: Configuration and Bursting. International Journal of Green Computing, 5(2), 2014. pdf
  • J-M. Salmela, P. Thanisch, O. Sotamaa, T. Niemi. Games and Energy: Profiling Power Usage During Play. In Lugmayr Artur, Franssila Helj, Paavilainen Janne ed. Proceedings of the 18th International Academic MindTrek Conference, Tampere, Finland, November 2014.
  • J. Kommeri and T. Niemi Energy-efficient Heterogeneous Cluster and Migration. ENERGY 2014, the Fourth International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies, April, 2014. pdf
  • Abdurachmanov, D., Elmer. P., Eulisse, G., Knight, R., Niemi, T.P., Nurminen. J.K., Nyback, F., Pestana, G., Ou, Z. Techniques and tools for measuring energy efficiency of scientific software applications. 16th International workshop on Advanced Computing and Analysis Techniques in physics research (ACAT’2014), Prague, Czech Republic, September, 2014. pdf
  • J. Kommeri, T. Niemi, and O. Helin. The impact of workload on energy efficiency of virtualized systems. International Journal On Advances in Intelligent Systems, 5(3-4):441 - 450, 2012. pdf
  • T. Niemi and A.-P. Hameri. Memory-based Scheduling of Scientific Computing Clusters. The Journal of Supercomputing, 61(3):520-544, 2012.
  • T. Niemi, J. Kommeri, and A-P Hameri Improving Energy-Efficiency of Scientific Computing Clusters. Sustainable Green Computing: Practices, Methodologies and Technologies, IGI Global, 2012. pdf
  • M. Sevalnev, S. Aalto, J. Kommeri and T. Niemi. Using queueing theory for controlling the number of computing servers. ICGREEN 2012 (Third International Conference on Green IT Solutions, Geneva, Switzerland, July 2012. pdf
  • J. Kommeri, T. Niemi and M. Niinimaki. Study of virtualization energy-efficiency in high energy physics computing. ICGREEN 2012 (Third International Conference on Green IT Solutions, Geneva, Switzerland, July 2012.
  • J. Kommeri, T. Niemi, and O. Helin. Energy Efficiency of Server Virtualization. The Second International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies, ENERGY 2012, St. Maarten, Netherlands Antilles, March 25-30, 2012. pdf
  • T. Niemi, J. Kommeri, and A.-P. Hameri Using Multitasking and SSD Disks for Optimising Computing Cluster Energy-Efficiency. The First International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies, ENERGY 2011, Venice/Mestre, Italy, May 22-27, 2011.

Thesis

  • Gonzalo Pestana, "Energy Efficiency in High Throughput Computing: Tools, techniques and experiments," MSc thesis, September 2016 pdf
  • Mikael Hirki, “Energy and performance profiling of scientific computing,” MSc thesis, December 2015 pdf
  • Carolina Lindqvist, “ An alarm aggregation framework for critical IT Service monitoring at CERN”, MSc thesis pdf
  • Johan Strandman, “Distributed Computing as a Source of Heat: The Design and Evaluation of a Computerized Heater”, MSc thesis pdf
  • Filip Nybäck, “Improving the support for ARM in the IGProf profiler”, MSc thesis pdf

Personnel

Prof. Jukka K. Nurminen, Project leader
Jukka started as professor of computer science at Aalto University at the beginning of 2011. He has a strong industry background with almost 25 years experience of software research at Nokia Research Center. Jukka’s experience ranges from mathematical modeling to expert systems, from network planning tools to solutions for mobile phones, and from R&D project management to tens of patented inventions. Jukka received his M.Sc degree in 1986 and Ph.D. degree in 2003 from Helsinki University of Technology. His main interests are energy-efficient computing and communication, mobile peer-to-peer, smart traffic, distributed solutions for mobile devices, and web communications

Tapio Niemi, Project coordinator (tapio.niemi(a)cern.ch)
Dr. Tapio Niemi holds a PhD in Computer Science from the University of Tampere (2001). He worked in the University of Tampere as a scientist (1997-2001) and as an assistant professor (2002-2004). Since 2004 he has worked in the Technology Programme of the Helsinki Institute of Physics (HIP) at CERN. Dr. Niemi's research interests are green computing and data management. He has published over 60 refereed scientific articles both in international journals and conference publications.

Kashif Nizam Khan, PhD student
Mr. Kashif Nizam Khan obtained his Masters degree in Security and Mobile Computing as an Erasmus Mundus scholar From Aalto University, Finland and NTNU, Norway (2010). He is a faculty member (on leave) of Department of Computer Science and Engineering, Islamic University of Technology, Bangladesh. He is currently working with Helsinki Institute of Physics and Department of Computer Science, Aalto University towards his doctoral degree. His current research focus is on energy efficiency in data-centers and cloud environment focusing on scientific computing. He has published several scientific articles.

Ari-Pekka Hameri, Senior scientist, Professor at HEC-UNIL.
Efficient use of any production resources has been central to the research done by Prof. Hameri. He has published over 50 articles in international management and science journals concerning industrial IT, and management of production, projects and supply chains. The problem encountered with the high throughput and energy-efficient use of the computin resources is analogous to this previous research. He has been involved with numerous international research and spin-off projects dealing with industrial IT and production management. He is also directly involved with the development of scientific and industrial Grid computing applications.

Seppo Heikkilä, Scientist, CERN
Seppo graduated as a M.Sc. in "Electrical and Telecommunications Engineering" from Aalto University (former Helsinki University of Technology) and as a M.Sc. in "Space Studies" from International Space University (ISU) Strasbourg/France in 2005 and 2006, respectively. In 2011, he finished his Ph.D. on automation technology in a joint-project with European Space Agency (ESA) and Aalto University. At the beginning of 2012, Seppo arrived to CERN when he started to work with a secure multi-cloud storage software in the Helsinki Institute of Physics (HIP) technology programme. Between March 2013 and December 2015, he worked as a senior research fellow in the CERN openlab and the CERN IT Data Storage and Services (DSS) group, where his research was focused on cloud storage systems. He is currently working for Horizon 2020 SSICLOPS project.

Jukka Kommeri, PhD student
Mr. Jukka Kommeri has seven years of administration work experience with Linux systems both in the Computing Center of the Helsinki University of Technology and in the Helsinki Institute of Physics at CERN. His current research interest is in gathering and developing methods for improving energy-efficiency of cloud computing clusters by using virtualisation.He has published several scientific articles.

Alumni

Zhonghong Ou, PostDoc, Aalto University. (now Professor at BUPT)
Since 2010 Zhonghong has been a PostDoc at Aalto University. His has a wide spectrum of research interests, including: cloud computing platform, cloud storage systems, energy-efficiency of mobile platforms, vision-based wearable navigation system, Internet of Things, Peer-to-Peer networks

Gonzalo Pestana, MSc thesis student, Aalto University
Gonzalo is working on his MSc thesis on heterogeneous hardware for scientific software

Mikael Hirki, MSc thesis student, Aalto University
Mikael is working on his MSc thesis on energy profiling of scientific software

Filip Nybäck, MSc thesis student, Aalto University
Filip is working on his MSc thesis on energy profiling of scientific software

Intranet

Topic attachments
I Attachment History Action Size Date Who Comment
PDFpdf EEUI.pdf r1 manage 1152.7 K 2017-08-08 - 12:28 TapioNiemi  
PDFpdf EnrgyIGProf.pdf r1 manage 205.0 K 2015-06-12 - 12:15 JukkaNurminen  
PDFpdf HeatingPaper02_tn.pdf r1 manage 1070.6 K 2015-06-12 - 12:05 JukkaNurminen CCGrid 2015 paper on Computing Heaters
PDFpdf Kimmerlin_Practical_2017.pdf r1 manage 538.5 K 2017-08-28 - 14:31 SeppoHeikkilae  
PDFpdf Nodari2016Inventory.pdf r1 manage 116.2 K 2016-03-29 - 15:31 JukkaNurminen  
PDFpdf OM_computing_centers.pdf r1 manage 484.6 K 2016-02-09 - 10:33 TapioNiemi  
PDFpdf cooldc-16-instruction-decoders-final-version-second-submission.pdf r1 manage 178.6 K 2016-03-29 - 15:41 JukkaNurminen  
PDFpdf dbEEBurstiness.pdf r1 manage 931.3 K 2016-01-12 - 14:58 JukkaKommeri IJGC 2014 paper on db burstiness
PDFpdf energy2012.pdf r1 manage 279.9 K 2015-06-26 - 10:09 TapioNiemi energy2012
PDFpdf energy2012_journal.pdf r1 manage 682.5 K 2015-06-26 - 10:25 TapioNiemi  
PDFpdf energy2014.pdf r1 manage 439.4 K 2015-06-15 - 12:43 TapioNiemi  
PDFpdf eu-ssiclops-oulu.pdf r2 r1 manage 283.0 K 2017-11-13 - 13:15 SeppoHeikkilae  
PDFpdf green-big-data-cern.pdf r1 manage 359.2 K 2017-11-13 - 11:51 TapioNiemi  
PDFpdf icgreen.pdf r1 manage 597.1 K 2016-04-18 - 11:13 JukkaNurminen  
PDFpdf icgreen12_sevalnev.pdf r1 manage 120.7 K 2015-06-26 - 10:05 TapioNiemi icgreen12paper
PDFpdf migraatioET_SC.pdf r1 manage 1223.4 K 2016-10-17 - 11:48 TapioNiemi  
PDFpdf niemi_igi_final.pdf r1 manage 208.2 K 2016-01-11 - 15:10 TapioNiemi  
PDFpdf sunburn.pdf r1 manage 2009.4 K 2017-08-08 - 12:17 TapioNiemi  
Edit | Attach | Watch | Print version | History: r34 < r33 < r32 < r31 < r30 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r34 - 2017-11-13 - SeppoHeikkilae
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback