KVM Parametrisation

Aim

Look closer into performance issues with KVM. Get hands on a RH6 beta, check benchmarking, check kernel options, check things like cpu affinity, and eventually as well take a vanilla kernel and a upstream KVM and see if that helps.

Requirements: Knowledge Linux OS and Virtualisation technology (KVM hypervisor)

Data sources

Information	Source	Description

tbd.

Current tools

Tool	Type	Executed on	Provided by

Here is a list of the tools used:

• CPU Benchmark: HEPSPEC06
• Disk IO Benchmark: IOZONE
• Network Benchmark: IPERF 2.0.4

References

Priv.Com. with A. Chierici at HEPiX 2010/Cornell

http://117.103.105.177/MaKaC/sessionDisplay.py?sessionId=52&slotId=0&confId=3#2010-10-19

http://indico2.twgrid.org/MaKaC/contributionDisplay.py?contribId=89&sessionId=31&confId=3

Link to the lxcloud coordination meeting

http://indico.cern.ch/conferenceDisplay.py?confId=114145

http://indico.cern.ch/conferenceDisplay.py?confId=114486

http://indico.cern.ch/conferenceDisplay.py?confId=115569

http://indico.cern.ch/conferenceDisplay.py?confId=116475

Objectives

At the end of the project we should have

• an idea of how KVM behaves
• a set of recommendations for tuning to get optimal CPU, I/O and networking performance

Individual tasks

• Check performance of KVM in CPU, I/O and networking
• Checkout possibilities to tune KVM, including testing of newer version if required.
• Test these tuning options quantitatively.

Benchmarks on KVM

Test specifications

• The hardware used for the benchmark test is composed of one physical machine with 8 cores Intel(R) Xeon(R) and 24GB RAM.
• The VMs specs are: KVM VM 1 VCPU, 2GB RAM
• The OS installed:

Hypervisor OS: SLC release 5.5 (Boron) 2.6.18-194.11.3.el5.cve20103081 x86_64

VMs OS: SLC release 5.5 (Boron) 2.6.18-194.17.4.el5 x86_64

• KVM version: KVM 83

CPU Benchmark

• Tools: HEPSPEC06
• Scripts: spec2k6_vm.sh

---

The test is designed as follow:

Part I Without optimization of KVM

1.Start 8 VMs with 1 CPU running in a single hypervisor

2.Running the script on each VM at the same time without pinning CPU;

3.Pinning CPU for each VM, then running the script on each VM.

Part II Optimization of KVM with ‘modprobe kvm-intel enable_ept=0’

1.Modify the file: /etc/modprobe.conf on hypervisor

[root@lxbsq0910 ~]# cat /etc/modprobe.conf

alias eth0 igb

alias eth1 igb

alias scsi_hostadapter ahci

options kvm-intel enable_ept=0 (add this line)

options edac_mc log_ue=1 check_pci_parity=1

2.Reboot the hypervisor

3. Start 8 VMs with 1 CPU running in the hypervisor

4. Running the script on each VM at the same time without pinning CPU;

5. Pinning CPU for each VM, then running the script on each VM.

---

Test Results

You can read the attach file

Disk IO Benchmark

• Tools: IOZONE
• Options: -Mce -I -+r -r 256k -s 8g -f /usr/vice/cache/iozone_$i.dat$$ -i0 -i1 -i2

The test is designed as follow:

1.Running 8 IOZONE processes in the hypervisor;

2.Start 8 VMs in the hypervisor and run IOZONE test at the same in each VM under Pinning and UnPinning.

---

Test Results

You can read the attach file

Network Benchmark

• Tools: IPERF 2.0.4
• Options: ‘-p 11522 -w 458742 -t 60 ‘ means TCP window size is 256KB, the test duration time is 60 secs(default is 10 secs)
• Physical Server: lxbsq0910
• VM Servers: vmbsq091000~ vmbsq091007
• Client: lxvmpool005

• Server side command:iperf -s –p PortNumber -w 458742 -t 60
• Client side command: iperf -c ServerIP -p PortNumber -w 458742 [–P 8 ] b-t 60 (Port number should be same with the one on server side)

The test is designed as follow:

Part I

1.Set 8 parallel threads running at the same time on the client to test the hypervisor throughput performance;

2.Start 8 VMs running on the hypervisor and make them acted as servers. On the client side, I run 8 threads almost at the same time to connect each server respectively.

Part II

1.running a thread on the client to test the hypervisor throughput performance;

2.Doing 8 rounds separately. First round, start 1 process to connect 1 VM, Second, start 2 processes to connect 2 VMs respectively. Finally, start 8 processes to connect 8 VMs repectively.

---

Test Results

You can read the attach file

Conclusion

From the benchmark tests, we may draw a conclusion:

• For CPU benchmark, the pinning gets better performance than unpinning. And the performance can be improved 6%~8% by optimizing the KVM with disabling ept option. In order to get the best result, we’d better combine the pinning and disabling ept option. With this we can achieve an optimistic level that is the performance penalty is about 3%.
• For disk IO access, the write performance penalty is about 10%. While read performance loss is about 12% which is a little more than the write one.
• For network performance penalty in VMs is about 3%, It’s really optimistic. And the second picture shows the performance in VMs is nearly equal to the physical machine. What’s more, with 4 VMs get better value than the bare one. However, it’ s a preliminary test results, we should do more study to investigate and tune some parameters to optimize the network performance using real application.

These tests have been given us some information for the version of KVM. However, there are various kinds of VMs and kernels. Since each of them uses different type of techniques, the result might not be reproduced under other types of environment. We should do more tests to understand the performance impact of virtualization.

Reports

Week 08/11/2010

08/11/2010 - todo

-- JuanManuelGuijarro - 05-Nov-2010

• Benchmark_on_VMs.docx: It's a report of benchmark on CPU, Disk IO and Network performance.

• Benchmark_on_VMs.docx: It's a report of benchmark on CPU, Disk IO and Network performance.

• CPU_benchmark_test2.pptx: The slides about the CPU benchmark.

• Disk_IO_and_Network_Benchmark_on_VMs.pptx: The slides about the Disk IO and Network benchmark.

• rawdata.rar: Raw data of KVM performance test