Friday, May 8, 2020

Linux EC2 to On-Premise migration | a.k.a. The offline rsync method


This guide assumes that we have one EC2 instance with only the root volume i.e. one mount only. If you have multiple mounts you will need to adjust the steps accordingly and rsync the multiple mounts. We will be migrating the instance from AWS to a VMware clsuter however by adjusting the steps you can use this method to migrate to any other custom environment, even VirtualBox.

1. Preparation

  1. Source EC2 to be cloned (I name this SOURCE-VM)
  2. Create helper VM in VMware (I name this HELPER-VM)
  3. Create helper EC2 VM in AWS (I name this HELER-EC2)
  4. Create blank VM at the target VMware Site with a similar structure as the original (I will name this CLONE-VM). Make sure that the disk is about the same size and thin provisioned
  5. Take note of the disks location of the CLONE-VM.
    [WorkloadDatastore] 2f6ba65a-6af4-6f09-2666-124516e7c87c/debian.vmdk

Gather SOURCE-VM critical system information

cat /etc/*release*
uname -a

# Check if it has a package manager.
# It may be a customized distro and not have a package manager
Execute apt or yum - see if it produces any output

# See what is inside the /boot directorty
ls -la /boot

cat /proc/cpuinfo #Check CPU 
cat /proc/meminfo #Check Memory

#Check mounts and partitions
df -h 
cat /etc/fstab
cat /etc/mtab

# Check running processes
ps auxwff
netstat -tulnp

2. In AWS

  1. Take snapshot of the root volume of the SOURCE-VM to be cloned
  2. Attach the disk from the SOURCE-VM snapshot to the helper HELER-EC2
  3. Mount the disk into /mnt
  4. Verify that the contents from the attached disk match the contents from your SOURCE-VM. They need to contain the entire disk layout.

3. In VMware

Attach the empty disk that you have taken note before from the CLONE-VM to the HELPER-VM as an additional disk. Mount it to /mnt.

NOTE: You don't need to detach the disk from the CLONE-VM in order to attach it to another VM. In VMware the disks can be attached to multiple VMs at once.


Format the new drive attached from the CLONE-VM
# Identify the new disk
ll /dev/sd*

## /dev/sdd in my case

## This creates the partition and prepares it for ext4 fs
select /dev/sdd
mklabel msdos
mkpart primary ext4 0% 100%
set 1 boot on

## This formats the partition. Be very carefull not to use a wrong partition here, or it will wipe out everything.
mkfs.ext4 /dev/sdd1

## Mount the partition in /mnt
mount /dev/sdd1 /mnt/

4. Initiate data transfer

From the HELPER-EC2 initiate a data transfer of the contents of the root partition of the SOURCE-VM to the disk of the CLONE-VM that is attached to the HELPER-VM

Example in my case:
rsync -avxHAWX --numeric-ids --info=progress2 -e ssh /mnt/ root@

Once the rsync is finished, shutdown the HELPER-VM and detach the additional hard disk from the it.

5. Re-Install the bootloader

We now have almost everything we need for our CLONE-VM to start working, except that it will not boot without a bootloader.

To do that, we need to to a couple of things:

  • Remove any existing bootloader from AWS;
  • Remove any folders that contain additional options for booting. These options are tailored for use for an EC2 and may cause problems with our VM;
  • Reinstall the boatloader (GRUB);
  • Check and adjust /etc/fstab to match our new system layout;
  • As an additional cosmetic we also need to remove cloud-init.

To do all of this we need a temporary environment to work on. The system in this case was debian so I will use a bootable Ubuntu CD and go to recovery mode. Then we shall execute a chroot environment into our new disk.

The following is an example of commands used to identify the current boot loader. Then I am removing it. Then I remove any existing folders that have additional boot options. Finally I install the new bootloader. If you will do this on a CentOS/RedHad the commands are slightly different. Use Google find them out.

## Find any existing GRUB software and remove all of it.
$ apt list --installed | grep grub*

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

grub-common/now 2.02~beta2-36ubuntu3.15 amd64 [installed,upgradable to: 2.02~beta2-36ubuntu3.17]
grub-gfxpayload-lists/xenial,now 0.7 amd64 [installed,automatic]
grub-legacy-ec2/now 17.1-46-g7acc9e68-0ubuntu1~16.04.1 all [installed,upgradable to: 17.2-35-gf576b2a2-0ubuntu1~16.04.2]
grub-pc/now 2.02~beta2-36ubuntu3.15 amd64 [installed,upgradable to: 2.02~beta2-36ubuntu3.17]
grub-pc-bin/now 2.02~beta2-36ubuntu3.15 amd64 [installed,upgradable to: 2.02~beta2-36ubuntu3.17]
grub2-common/now 2.02~beta2-36ubuntu3.15 amd64 [installed,upgradable to: 2.02~beta2-36ubuntu3.17]

$ apt-get remove grub && apt-get purge grub #The 2nd removes any grub configuration

$ rm -f /etc/default/grub
$ rm -rf /etc/default/grub.d ## This folder has AWS specific configuraiton thus it will cause problems later on and must be removed.

## Reinstall GRUB
$ apt update
$ apt install grub2 os-prober

At this point GRUB will ask you to which drive the boot sectors to be installed on. Choose the whole drive /dev/sda and not the partition /dev/sda1.

After this the system should boot normally.


No comments:

Post a Comment