This is neat technology. The other side of the coin is that you may allocate a thick provisioned resource but at a fraction of its entirety. What does this mean? The short term -- skimping -- I use this when I can't estimate the exact requirement of an application. I rely on the "hot add" technology of virtualization to allocate the final pieces of the architecture when more accurate information is available. In the virtualization lingo, the terms "hot add" and "hot plug" are often used interchangably and likely pertain to the same thing.
Modern operating systems play a big role in this "hot add" technology. For example, in the Linux world once the application has been tested and it was later found to be CPU-bound, just hot add more cores. Not really that interesting, right? But wait, you can do this without having to reboot the virtual machine -- NO DOWNTIME! Adding virtual CPUs while it is running -- now that's cool stuff! This of course assumes that the hot add feature has been turned on.
This is how its done on the VMware side of things.
The same can be done to memory -- once a guest is identified to be memory-bound -- just hot add more memory.
A recent experience taught me that this doesn't always work. It works for CPU hot plug, but for memory hot add it doesn't. I'm using the recent version of RHEL 6.4 x86_64 and CentOS 6.4 x86_64. Both of these operating systems don't work well with memory hot add, but their older versions used to work.
Executing a memory hot add on a RHEL/CentOS 6.4 guest, this error appears:
------------[ cut here ]------------ WARNING: at arch/x86/mm/init_64.c:701 arch_add_memory+0xe2/0x100() (Not tainted) Hardware name: VMware Virtual Platform Modules linked in: vsock(U) vmci(U) autofs4 sunrpc ipv6 ipt_REJECT ppdev parport_pc parport microcode vmware_balloon e1000 shpchp sg i2c_piix4 i2c_core ext4 mbcache jbd2 sd_mod crc_t10dif sr_mod cdrom mptsas mptscsih mptbase scsi_transport_sas pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod [last unloaded: nf_conntrack] Pid: 25, comm: kacpi_notify Not tainted 2.6.32-358.2.1.el6.x86_64 #1 Call Trace: [] ? warn_slowpath_common+0x87/0xc0 [ ] ? warn_slowpath_null+0x1a/0x20 [ ] ? arch_add_memory+0xe2/0x100 [ ] ? add_memory+0xb7/0x1c0 [ ] ? acpi_memory_enable_device+0x95/0x12b [ ] ? acpi_memory_device_add+0x118/0x121 [ ] ? acpi_device_probe+0x50/0x122 [ ] ? driver_probe_device+0xa0/0x2a0 [ ] ? __device_attach+0x0/0x60 [ ] ? __device_attach+0x0/0x60 [ ] ? __device_attach+0x53/0x60 [ ] ? bus_for_each_drv+0x64/0x90 [ ] ? device_attach+0xa4/0xc0 [ ] ? bus_probe_device+0x2d/0x50 [ ] ? device_add+0x527/0x650 [ ] ? pm_runtime_init+0xcb/0xe0 [ ] ? device_register+0x1e/0x30 [ ] ? acpi_add_single_object+0x837/0x9e8 [ ] ? acpi_ut_release_mutex+0x63/0x67 [ ] ? acpi_bus_check_add+0xe0/0x138 [ ] ? acpi_os_execute_deferred+0x0/0x36 [ ] ? acpi_bus_scan+0x3a/0x71 [ ] ? acpi_bus_add+0x2a/0x2e [ ] ? acpi_memory_device_notify+0xa6/0x24f [ ] ? acpi_os_execute_deferred+0x0/0x36 [ ] ? acpi_bus_get_device+0x2a/0x3e [ ] ? acpi_bus_notify+0x4b/0x82 [ ] ? acpi_os_execute_deferred+0x0/0x36 [ ] ? acpi_ev_notify_dispatch+0x64/0x71 [ ] ? acpi_os_execute_deferred+0x29/0x36 [ ] ? worker_thread+0x170/0x2a0 [ ] ? autoremove_wake_function+0x0/0x40 [ ] ? worker_thread+0x0/0x2a0 [ ] ? kthread+0x96/0xa0 [ ] ? child_rip+0xa/0x20 [ ] ? kthread+0x0/0xa0 [ ] ? child_rip+0x0/0x20 ---[ end trace 081e3b980cb5c943 ]--- ACPI:memory_hp:add_memory failed ACPI:memory_hp:Error in acpi_memory_enable_device acpi_memhotplug: probe of PNP0C80:00 failed with error -22 driver data not found ACPI:memory_hp:Cannot find driver data
You will see the above error message when the command "dmesg" is executed on a terminal window. This seems to be a bug in the kernel module for memory hot add.
This is a bug that I hope gets fixed in future versions of the Linux kernel. If you rely on memory hot add feature, please be aware of this limitation on the new versions of RHEL and CentOS. It seems to be rooted to the kernel code so most likely all other flavors of Linux are in the same boat.
The Windows operating system doesn't seem to have this problem. Hot plug of virtual CPUs and hot add of memory works without any problem.
