General Information

LVM provides a way to create virtual disks and containers in order to provide flexibility with parition sizes and extending/shrinking space without shutting a system down.

Checklist

• Distro(s): Any
• Other: The hard disk being formatted is not in use/blank. These steps will erase all data on the target disk.

Whether expanding or creating new storage, the summarized steps are:

• Add virtual disk to VM
• Scan SCSI Bus (if needed)
• Format the Disk with a partition
• Create physical volume using the new disk's partition

Add or expand the disk to the system either physically or virtually if a VM.

If adding a disk to a virtual machine and the new disk does not show up:

One Liner Method

echo "- - -" > /sys/class/scsi_host/$(grep mpt /sys/class/scsi_host/host?/proc_name | grep -o -E '(host[0-9])')/scan

Manual Method

• Find the host bus number

  grep mpt /sys/class/scsi_host/host?/proc_name

  • Example return value:

    /sys/class/scsi_host/host2/proc_name:mptspi

  • “host2” is the important value
• Used the returned host# to scan the SCSI bus

  echo "- - -" > /sys/class/scsi_host/host#/scan

  • Example from returned value

    echo "- - -" > /sys/class/scsi_host/host2/scan

• The hyphens represent controller,channel,lun, so – - – indicates all controllers, all channels, and all luns should be scanned.

If you have expanded an existing disk, rescan the scsi bus

echo 1 > /sys/class/scsi_device/<device>/rescan

Where “<device>” is your disk such as “sda”.

/dev/sdb is used in this example.

Use gdisk in order for GPT instead of MBR. (seriously, don't use fdisk…let MBR die already)

gdisk /dev/sdb

Once in gdisk:

• Use option “o” to create a new empty GUID Partition table. (GPT)
• “w” to write table entries

• Go back into gdisk. (gdisk /dev/sdb)
• “n” to create a new partition
  • Choose the defaults for partition number, start, and end blocks to fill the entire disk
  • Use code “8e00” for Linux LVM
• “w” to write changes and exit

About: Think of physical volumes as your “virtual disks” to use in LVM.

Create a physical volume using the parition that the LVM is on.

pvcreate <device1-path> [device2-path] [deviceX-path]

Example:

pvcreate /dev/sdb1

Continue on to either Expanding Storage or Creating New Storage.

Expanding storage steps.

To expand storage for a logical volume, the summarized steps are:

Performed above in “Expanding or New”:

• Add virtual disk to VM
• Re-Scan SCSI Bus (if needed)
• Format the Disk with a partition
• Create physical volume using the new disk's partition

Remaining Steps:

• Extend the volume group using the new physical volume
• Extend the logical volume
• Verify new LVM layout
• Grow the file system
• Cleanup (if a pvmove of the logical volume was done to a new disk)
  • Verify old physical volume is not used
  • Reduce the volume group by removing the physical volume
  • Remove the physical volume LVM label
  • Delete virtual disk from virtual management UI

1. Add the new physical disk to the existing volume group

   vgextend vglocal /dev/sdb1

1. Extend the logical volume that needs the space (/home will be used in this example)
   1. Option 1(Preferred): Move the physical extents of the logical volume to the new disk, so all of it resides on the same VMDK, and then use all of that new disk's space.
      1. There may be a performance hit if a logical volume is spanned across multiple VMDKs, so this method is preferred.

         pvmove --name lvhome /dev/sda2 /dev/sdb1
         lvextend --resizefs /dev/vglocal/lvhome /dev/sdb1

   2. Option 2: Give the logical volume all of the space from the newly added physical volume

      lvextend --resizefs --extents +100%PVS /dev/vglocal/lvhome /dev/sdb1

   3. Option 3: Give the logical volume a specific amount of additional free space from a specific physical volume

      lvextend --resizefs --size +10G /dev/vglocal/lvhome /dev/sdb1

   4. Option 4: Give the logical volume all of the free space available to the volume group (potentially across multiple physical volumes)

      lvextend --resizefs --extents +100%FREE /dev/vglocal/lvhome

Extending a swap volume is slightly different, as the “–resizefs” flag will not work.

1. Example: Specify the total size of the logical swap volume

   lvextend --size 8G /dev/vglocal/lvswap /dev/sdb1

2. Disable all swap

   swapoff -a

3. Create a new swap area, overwriting the original

   mkswap /dev/vglocal/lvswap

4. Enable swap

   swapon -a

5. Verify

   swapon -s

1. Verify LVM Allocation
   1. Logical Volume:

      lvs

   2. Volume Group:

      vgs

   3. Physical Volumes:

      pvs

Logical Volume Location(s) on Physical Volumes

If you would like to see what logical volumes reside where, a nice command to view physical extent to logical volume mappings is:

pvdisplay -m

Skip if you used the “–resizefs” flag to lvextend above.

1. Grow the file system
   1. Ext2/3/4

      resize2fs /dev/vglocal/lvhome

   2. XFS

      xfs_growfs /dev/vglocal/lvhome

2. Verify filesystem space

   df -h

If you have just moved all of the data of a logical volume off of an old physical volume, the old physical volume can go away if it is no longer in use.

Examples below: You have just moved everything off of /dev/sdc1 to a new disk and can now remove the old disk (/dev/sdc).

pvs -o +pv_used

vgreduce vgname olddisk

vgreduce vglocal /dev/sdc1

pvremove olddisk

pvremove /dev/sdc1

Login to the virtualization user interface and delete the old disk from inventory.

Errors During LVM Commands

• If you see errors during LVM commands like this after a device delete from vmware

  root@llsrlscd01 home $ pvs
    /dev/sdc: read failed after 0 of 4096 at 0: Input/output error
    /dev/sdc: read failed after 0 of 4096 at 17179803648: Input/output error
    /dev/sdc: read failed after 0 of 4096 at 17179860992: Input/output error
    /dev/sdc: read failed after 0 of 4096 at 4096: Input/output error
    /dev/sdc1: read failed after 0 of 4096 at 17178755072: Input/output error
    /dev/sdc1: read failed after 0 of 4096 at 17178812416: Input/output error
    /dev/sdc1: read failed after 0 of 4096 at 0: Input/output error
    /dev/sdc1: read failed after 0 of 4096 at 4096: Input/output error
    PV         VG      Fmt  Attr PSize   PFree
    /dev/sda2  vglocal lvm2 a--   19.50g 3.50g
    /dev/sdd1  vglocal lvm2 a--  250.00g    0
    /dev/sde1  vglocal lvm2 a--   40.00g    0

  • Fix by rebooting the system or

    echo 1 > /sys/block/sdX/device/delete

    • Where “sdX” is the device, such as 'sdc' in the above example.

To create new storage for a logical volume, the summarized steps are:

Performed above in “Expanding or New”:

• Add virtual disk to VM
• Re-Scan SCSI Bus (if needed)
• Format the Disk with a partition
• Create physical volume using the new disk's partition

Remaining Steps:

• Create a volume group
• Create a logical volume
• Create a file system on the logical volume
• Mount the logical volume

About: Volume groups aggregate physical volumes into a usable pool of disk space.

vgcreate <vgname> <physical-device>

Example:

vgcreate vgstorage /dev/sdb1

About: Logical volumes use the disk space made available via a volume group.

lvcreate –size <size> –name <lv-name> <vg-name>

Example: Create a 100G logical volume

lvcreate --size 100G --name lvbackups vgstorage

Example: Create a logical volume that uses 100% of the volume group free extents

lvcreate --extents 100%FREE --name lvbackups vgstorage

About: So far, the only things being created were pretty much containers. Create the actual file system below.

mkfs -t <fstype> -L <label> /dev/mapper/<vgname>-<lvname>

Example:

mkfs -t ext4 -L Backups /dev/mapper/vgstorage-lvbackups

blkid
/dev/mapper/vgstorage-lvbackups: UUID="d89744ad-133c-452b-98d8-9480ef18fe77" TYPE="ext4"

vim /etc/fstab
UUID=d89744ad-133c-452b-98d8-9480ef18fe77 /opt/backups ext4 defaults 0 2

mount -a
df -h

We don't always get a name right the first time. Luckily, LVM allows for easy renaming of logical volumes and volume groups.

Rename a Volume Group (vgrename vg-oldname vg-newname)

vgrename vgbackupmirror vgredusb

Rename a Logical Volume (lvrename vgname lv-oldname lv-newname)

lvrename vgredusb lvbackupmirror lvredbackups

LVM allows you to “freeze” the metadata of files. This allows for things like rsync to complete successfully even if files it is trying to sync are modified during the operation.

{TODO}