Nested virtualization allows you to run a virtual machine (VM) inside another VM [1]. Both Intel and AMD supports nested virtualization.

This is very helpful when you are experimenting with the hypervisor related technologies. Example: I will be able to run KVM and Virtualbox both on my laptop but in different VMs. Also I will be able to run local installation of imagefactory to build Vagrant images in a VM  as imagefactory need a hypervisor to run the build . The best part is, I can experiment with all of these inside different VMs without damaging my primary workstation’s hypervisor.

The below steps are done on a Fedora 23 running a Lenovo Thinkpad with Intel chipset.

Step 1: Make sure Intel virtualization (VT) is enabled for the host machine.

$ cat /proc/cpuinfo | grep vmx

flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts

The output should contain vmx else Intel virtualization (VT) is not enabled on the machine. You should first fix the setting in the BIOS.

Step 2: Install KVM on the F23 host.

$ dnf install @virtualization

Nested virtualization should be disabled bydefault

$ cat /sys/module/kvm_intel/parameters/nested
 N

Step 3: Enable nested virtualization.  Run below commands as root

• Temporarily remove the kvm kernel module

    $ sudo rmmod kvm-intel

• Add the following directive to /etc/modprobe.d/dist.conf

  $ sudo sh -c "echo 'options kvm-intel nested=y' >> /etc/modprobe.d/dist.conf"

• Insert the kvm module back in the kernel

   $ sudo modprobe kvm-intel

There is alternative way to do the same i.e. pass  kvm-intel.nested=1 on kernel commandline [3]

Step 4: Reboot and verify that nested virtualization is enabled

• Check that nested virt is enabled

$ sudo cat /sys/module/kvm_intel/parameters/nested
 Y

Step 5: Install the beefy VM. (Lets call it parent VM)

• I used CentOS 7 minimal ISO i.e. CentOS-7-x86_64-Minimal-1503-01.iso to install the VM through Virtual Machine Manger.
• Parent VM configuration : 50GB disk, 4GB RAM and 4 vCPUs

Step 6: Enable the VM to use nested virt

• Go to -> Virtual Machine Manger GUI -> CPU properties -> select “Copy host CPU configuration”

There is also another option i.e. host-passthrough [1] . It is supposed to be more stable then “Copy host CPU configuration” but I have not tried that yet.

Step 7:  Check that Intel virtualization (VT) is enabled in the VM

$ cat /proc/cpuinfo | grep vmx

Step 8: Install KVM inside the VM  [4]

$ yum install qemu-kvm qemu-img
$ yum install libvirt libvirt-python python-virtinst

$ systemctl enable libvirtd
$ systemctl start libvirtd
$ systemctl status libvirtd

Step 9:  Install the child VM inside the parent VM

• I used Virtual Machine Manger to connect to the parent VM and then install the child VM.
• Used the same CentOS 7 minimal ISO i.e. CentOS-7-x86_64-Minimal-1503-01.iso to install  the child VM.

[1] https://fedoraproject.org/wiki/How_to_enable_nested_virtualization_in_KVM

[2] http://docs.openstack.org/developer/devstack/guides/devstack-with-nested-kvm.html

[3] http://kashyapc.com/2012/01/14/nested-virtualization-with-kvm-intel/

[4] https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Host_Configuration_and_Guest_Installation_Guide/chap-Virtualization_Host_Configuration_and_Guest_Installation_Guide-Guest_Installation.html