HOW-TO: P2V (Physical to Virtual) Prep Work for Ubuntu

This post is supplementary to the previously written article Prep Work for Linux P2V (Physical to Virtual). The previous post covered mostly the tasks for an RPM based host (RedHat, Fedora and CentOS). This time it will cover more of the Debian based distribution.

It was just recently that I worked on several Ubuntu hosts. These presented a different perspective on the P2V process, especially since I had been tasked to convert them with little time to spare -- it is a spur of the moment, unexpected, unplanned need. With the added adjectives, you probably get the idea.

Although Linux-based, Ubuntu is a bit different than its RPM based brothers. For one, Ubuntu disk devices are mounted based on UUID. That is just the primer. The end of the P2V of VMware vCenter Standalone Converter prompted an error installing grub. To give you an idea of the error, refer to the screenshot below.

P2V Grub Install Error

FAILED: An error occurred during the conversion: 'GrubInstaller::InstallGrub: /usr/lib/vmware-converter/installGrub.sh failed with return code: 127, and message: Installing GRUB1 on (hd0)... /vmware-updateGrub.sh: 52: /vmware-updateGrub.sh: grub: not found Error installing GRUB Command: grub --no-floppy --batch --device-map="/vmware-device.map" root (hd0,0) setup (hd0) Error running vmware-updateGrub.sh through chroot into /mnt/p2v-src-root /usr/lib/vmware-converter/installGrub.sh: line 143: /mnt/p2v-src-root:
Is a directory '

Let's tackle these prep work one at a time.

For the UUID based filesystems, if you take a look at /etc/fstab you would see something similar to the below entries:

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
#                
# / was on /dev/sda1 during installation
UUID=a4fc19b3-6df9-4fcb-a565-12a9bd59fff1 /               ext4    errors=remount-ro 0       1
# swap was on /dev/sda5 during installation
UUID=1061e56a-f69d-4d44-b3d1-566c960719b1 none            swap    sw              0       0

The operating system is good enough to note the disk slices and its corresponding UUID unique identifiers. However, if this is not present on your system, the command "blkid" will show the same information.

[admin@vhost ~] sudo blkid
/dev/sda1: UUID="a4fc19b3-6df9-4fcb-a565-12a9bd59fff1" TYPE="ext4" 
/dev/sda5: UUID="1061e56a-f69d-4d44-b3d1-566c960719b1" TYPE="swap" 

The above entries need to be converted to its /dev/sdX counterpart prior to P2V. This is a simple change of removing the UUID=xxxxx-xxx-xxx-xxx and replacing it with its /dev/sdX equivalent. Exercise prudence, and make a backup of the file /etc/fstab prior to making changes.

Next, let's deal with the GrubInstall error. The grub install happens when the P2V is almost complete, meaning, the copying of the physical host files are done. The only challenge is the virtual machine made will not boot. This was a tough one to resolve at first.

As it turns out, you require the Linux Security Remix ISO file. Boot off the ISO and execute Boot Repair (recommended Boot Repair).

Once Boot Repair is done, reboot and the virtual machine is up and running.

The only hiccup that delayed the activity for me was to download the Linux Security Remix ISO. It is close to 1GB in size so if you are doing a lot of P2V conversions this file will come in handy. There are two (2) versions of this file -- the 32-bit and the 64-bit versions. If your system is 64-bit, the 32-bit version will not work. Having both versions is recommended.

As consolation, the only similarity I found with other Linux flavors was on the file /etc/udev/rules.d/70-persistent-net.rules, where the MAC addresses of the physical machine are hard-coded. If this file exists, comment out the entries or remove the file completely prior to P2V. The udev process will re-create this file on boot-up.

Share:

ERROR: "The operation is not allowed in the current state of the host"

I experienced the above error while powering on virtual machines after a scheduled shutdown. Just so you can replicate my experience, I was administering vCenter v5.5 Update 2b and ESXi v5.5 Update 2 hosts. The cluster's HA was turned off, DRS was set to partially automated from its fully automated setting, and all ESXi hosts were put into maintenance mode prior to them shutting down.

Powering up there were no issues -- all VMFS datastores were mounted and there are no alarms nor alerts. So after checks and everything was all set, ESXi hosts were exited from maintenance mode and virtual machines were started.

Most of the virtual machines booted back. But some of those that were powered up did not proceed on booting but instead prompted the error: "The operation is not allowed in the current state of the host."

This was completely new to me. And VMware forums and public knowledge base had an almost similar error: "The operation is not allowed in the current connection state of the host." Which was not quite what I was facing.

I was using the vSphere Web Client, so I tried using the old graphical user interface. Things did not seem to change -- same error.

I restarted the VMware VirtualCenter Server service but I still ended up with the same error. So I started to scratch my head. Of the several virtual machines that manifested this error, one thing was common: they belonged to similar ESXi hosts.

I went back to reading the posts with the almost identical error and it suggested to restart the management agents on the ESXi hosts. So I tried that but instead of just the management agents, I executed a stop and start of the entire services.sh.

Below are the traces of the execution.

~ # services.sh stop
Running hp-ams.sh stop
Terminating hp-ams-watchdog process with PID 35513 50567 50568
Stopping process 35479 35477 ...
hpHelper process is now stopped.
Running vmware-fdm stop
Stopping vmware-fdm:success
Running hp-mst.init stop
Unable to unload module mst: Module not found
Running xorg stop
Running sfcbd stop
This operation is not supported.
Please use /etc/init.d/sfcbd-watchdog stop
Running wsman stop
Stopping openwsmand
Running snmpd stop
Running sfcbd-watchdog stop
Running hpcru.init stop
Running hpilo.init stop
Running hpsmx.init stop
Running vpxa stop
watchdog-vpxa: Terminating watchdog process with PID 34561
vpxa stopped.
Running vobd stop
watchdog-vobd: Terminating watchdog process with PID 33332
vobd stopped
Running lacp stop
watchdog-net-lacp: Terminating watchdog process with PID 33542
Running memscrubd stop
memscrubd is not running
Running smartd stop
watchdog-smartd: Terminating watchdog process with PID 34425
smartd stopped
Running dcbd stop
watchdog-dcbd: Terminating watchdog process with PID 34370
Running cdp stop
watchdog-cdp: Terminating watchdog process with PID 34328
Running nscd stop
watchdog-nscd: Terminating watchdog process with PID 34283
Running slpd stop
Stopping slpd
Running storageRM stop
watchdog-storageRM: Terminating watchdog process with PID 34231
storageRM stopped
Running hostd stop
watchdog-hostd: Terminating watchdog process with PID 34191
hostd stopped.
Running vmfstraced stop
watchdog-vmfstracegd: PID file /var/run/vmware/watchdog-vmfstracegd.PID does not exist
watchdog-vmfstracegd: Unable to terminate watchdog: No running watchdog process for vmfstracegd
vmfstracegd is not running
Failed to clear vmfstracegd memory reservation
Running lbtd stop
watchdog-net-lbt: Terminating watchdog process with PID 34130
net-lbt stopped
Running sdrsInjector stop
watchdog-sdrsInjector: Terminating watchdog process with PID 34094
sdrsInjector stopped
Running rhttpproxy stop
watchdog-rhttpproxy: Terminating watchdog process with PID 34053
rhttpproxy stopped.
Running sensord stop
sensord is not running
Running hp-oem.init stop
Running usbarbitrator stop
watchdog-usbarbitrator: Terminating watchdog process with PID 33941
usbarbitrator stopped
Running DCUI stop
Disabling DCUI logins
VobUserLib_Init failed with -1
Running ntpd stop
Stopping ntpd
watchdog-ntpd: Terminating watchdog process with PID 33876
Connect to localhost failed: Connection failure
Running vsantraced stop
watchdog-vsantraced: Terminating watchdog process with PID 33807
vsantraced stopped
watchdog-vsantracedUrgen: Terminating watchdog process with PID 33836
vsantracedUrgen stopped
Persisting traces to /locker/vsantraces
~ #
~ # services.sh start
Running vsantraced start
Scratch partition is not backed by persistent storage
Storing traces to /vmfs/volumes/4dc4866f-040de4b6-1f88-0017a477f825/vsantraces
vsantraced started
vsantracedUrgen started
Running ntpd start
Connect to localhost failed: Connection failure
Starting ntpd
Running DCUI start
Enabling DCUI login: runlevel = 
VobUserLib_Init failed with -1
Running usbarbitrator start
usbarbitrator started
Running hp-oem.init start
Running sensord start
sensord started
Running rhttpproxy start
rhttpproxy started.
Running sdrsInjector start
sdrsInjector started
Running lbtd start
net-lbt started
Running vmfstraced start
VMFS Global Tracing is not enabled.
Running hostd start
Ramdisk 'hostd' with estimated size of 803MB already exists
hostd started.
Running storageRM start
storageRM started
Running slpd start
Starting slpd
Running nscd start
nscd started
Running cdp start
cdp started
Running dcbd start
dcbd started
Running smartd start
smartd started
Running memscrubd start
The checkPages boot option is FALSE, hence memscrubd could not be started.
Running lacp start
LACP daemon started
Running vobd start
vobd started
Running vpxa start
Connect to localhost failed: Connection failure
Running hpsmx.init start
Running hpilo.init start
Running hpcru.init start
Running sfcbd-watchdog start
Connect to localhost failed: Connection failure
Connect to localhost failed: Connection failure
Running snmpd start
Running wsman start
Starting openwsmand
Running sfcbd start
This operation is not supported.
Please use /etc/init.d/sfcbd-watchdog start
Running xorg start
Running hp-mst.init start
Running vmware-fdm start
Starting vmware-fdm:success

Running hp-ams.sh start
Starting hpHelper service...
~ # 

One thing to note here is that during the stop and start of services.sh vCenter server will complain about the host being disconnected. This is normal as the daemons handshaking and talking to each other lose connectivity.

After the stop and start was done, powering up of the virtual machines that had errors were successfully executed. Another point chalked to experience.

Share:

HOW-TO: Online Resizing of a MultiPath LUN

Online resizing of a multipath LUN is challenging. Try to wrap your head around the task at hand for a bit and understand the underlying principles behind this complex and challenging task. It is overwhelming! I have been in this particular scenario a lot and I know how it is to be on your shoes the first time.

But trust me, it may not be as daunting a task as you may think. Bear with me for a bit as I outline the steps needed to perform this task, successfully and without complications.

The previous article, Add LUNs to a Linux Host Without Reboot suggested the use of a well working script "rescan-scsi-bus.sh" via installation of the sg3_utils RPM. This task will do the same thing. So if this package is not installed, the script will probably be missing from your system as well. It would be a good idea to install it (yum -y install sg3_utils).

The first task on the list is to expand the LUN on the storage level. If you have an EMC storage, get to Unisphere and expand the LUN or the OnCommand System Manager for NetApp. On other storage systems, use the tools native to the storage to expand. Once that is done use rescan-scsi-bus.sh to rescan the LUNs.

For me (and some of you), "rescan-scsi-bus.sh -l" may already work. If not, use "rescan-scsi-bus.sh --forcerescan" to detect the new LUN sizes.

NOTE: The examples below will use the multipath device /dev/mpath/mpath165; and, /dev/sdat, /dev/sdh, /dev/sdap and /dev/sdd as its underlying devices. Its size is 190GB and we will add 100GB to it.

Next, check the devices relevant to the LUN. In this case -- mpath165. Change this with the multipath alias of your LUNs. Execute:

[root@vhost ~#] multipath -l mpath165
mpath165 (36006016098402e007cb4e6580ef1e311) dm-17 DGC,VRAID
[size=190G][features=1 queue_if_no_path][hwhandler=1 alua][rw]
  \_ round-robin 0 [prio=0][enabled]
   \_ 1:0:1:3 sdat 66:208 [active][undef]
   \_ 0:0:1:3 sdh  8:112  [active][undef]
  \_ round-robin 0 [prio=0][active]
   \_ 1:0:0:3 sdap 66:144 [active][undef]
   \_ 0:0:0:3 sdd  8:48   [active][undef]

The above output means there are four (4) paths composing the multipath device mpath165. They are /dev/sdat, /dev/sdh, /dev/sdap and /dev/sdd. The outputs on your system may vary.

Then, let the system detect the new size of the LUNs.

[root@vhost ~#] blockdev --rereadpt /dev/sdat
[root@vhost ~#] blockdev --rereadpt /dev/sdh
[root@vhost ~#] blockdev --rereadpt /dev/sdap
[root@vhost ~#] blockdev --rereadpt /dev/sdd
[root@vhost ~#] blockdev --getsz /dev/sdat
  608174080
[root@vhost ~#] blockdev --getsz /dev/sdh
  608174080
[root@vhost ~#] blockdev --getsz /dev/sdap
  608174080
[root@vhost ~#] blockdev --getsz /dev/sdd
  608174080

From the above commands, only getsz will provide an output. The outputs will be the same (due to the multipathing) and note them down for the commands we will execute later. If the outputs of "getsz" are not the same, something is wrong. Re-execute "blockdev --rereadpt" for that particular device.

Provided the outputs are all the same, proceed with the instructions below.

Next, dump the current multipath configuration and write the output to files -- mpath165.bak.
[root@vhost ~#] dmsetup table mpath165 | tee mpath165.bak
  0 398458880 multipath 1 queue_if_no_path 1 alua 2 2 round-robin 0 2 1 66:208 1000 8:112 1000 round-robin 0 2 1 66:144 1000 8:48 1000

Create a copy of the file "mpath165.bak" -- say "mpath165.new". Edit this file and replace the second number (398458880, which is the current size of the LUN) with the output of "blockdev --getsz" above.

(NOTE: Double check that the file mpath165.new has new values, where what used to be 398458880 should now be 608174080.)

This is the part where the LUN gets to be offline for a split second. It is imperative to stick these commands together. Or else..
[root@vhost ~#] dmsetup suspend mpath165; \
  dmsetup reload mpath165 mpath165.new ; \
  dmsetup resume mpath165

What actually happens in the above command is the multipath device mpath165 got suspended. Its configuration was replaced with the configuration contained in the file mpath165.new (or a resize was made). Then, the I/O operations were resumed. This was done quickly enough as if nothing happened.

Notify the kernel of a change of partition tables. Try partprobe first.
[root@vhost ~#] partprobe /dev/mpath/mpath165
-- or --
[root@vhost ~#] kpartx -a /dev/mpath/mpath165
[root@vhost ~#] kpartx -l /dev/mpath/mpath165

Now its time for LVM to take care of the rest of the resizing. When done right, this procedure can save you from unnecessary downtime due to multipath LUN resizing. Having known this procedure, resizing multipath LUNs will be just a routine for you.


Share:

Subscribe for Latest Update

Popular Posts

Post Labels

100gb (1) acceleration (1) acrobat (1) adblock (1) advanced (1) ahci (1) airdrop (2) aix (14) angry birds (1) article (21) aster (1) audiodg.exe (1) automatic (2) autorun.inf (1) bartpe (1) battery (2) bigboss (1) binance (1) biometrics (1) bitcoin (3) blackberry (1) book (1) boot-repair (2) calendar (1) ccleaner (3) chrome (5) cloud (1) cluster (1) compatibility (3) CPAN (1) crypto (3) cydia (1) data (3) ddos (1) disable (1) discount (1) DLNA (1) dmidecode (1) dns (7) dracut (1) driver (1) error (10) esxi5 (2) excel (1) facebook (1) faq (36) faucet (1) firefox (17) firewall (2) flash (5) free (3) fun (1) gadgets (4) games (1) garmin (5) gmail (3) google (4) google+ (2) gps (5) grub (2) guide (1) hardware (6) how (1) how-to (45) huawei (1) icloud (1) info (4) iphone (7) IPMP (2) IPV6 (1) iscsi (1) jailbreak (1) java (3) kodi (1) linux (28) locate (1) lshw (1) luci (1) mafia wars (1) malware (1) mapsource (1) memory (2) mikrotik (5) missing (1) mods (10) mouse (1) multipath (1) multitasking (1) NAT (1) netapp (1) nouveau (1) nvidia (1) osmc (1) outlook (2) p2v (2) patch (1) performance (19) perl (1) philippines (1) php (1) pimp-my-rig (9) pldthomedsl (1) plugin (1) popcorn hour (10) power shell (1) process (1) proxy (2) pyspark (1) python (13) qos (1) raspberry pi (7) readyboost (2) reboot (2) recall (1) recovery mode (1) registry (2) rename (1) repository (1) rescue mode (1) review (15) right-click (1) RSS (2) s3cmd (1) salary (1) sanity check (1) security (15) sendmail (1) sickgear (3) software (10) solaris (17) squid (3) SSD (3) SSH (9) swap (1) tip (4) tips (42) top list (3) torrent (5) transmission (1) treewalk (2) tunnel (1) tweak (4) tweaks (41) ubuntu (4) udemy (6) unknown device (1) updates (12) upgrade (1) usb (12) utf8 (1) utility (2) V2V (1) virtual machine (4) VirtualBox (1) vmware (14) vsphere (1) wannacry (1) wifi (4) windows (54) winpe (2) xymon (1) yum (1) zombie (1)

RANDOM POSTS