This is an old revision of the document!
Logical Volume Management (LVM)
General Information
LVM provides a way to create virtual disks and containers in order to provide flexibility with parition sizes and extending/shrinking space without shutting a system down.
Checklist
- Distro(s): Any
- Other: The hard disk being formatted is not in use/blank. These steps will erase all data on the target disk.
Expanding or New
Whether expanding or creating new storage, the summarized steps are:
- Add virtual disk to VM
- Scan SCSI Bus (if needed)
- Format the Disk with a partition
- Create physical volume using the new disk's partition
Add/Expand Disk
Add or expand the disk to the system either physically or virtually if a VM.
Add Disk: Scan SCSI Bus
If adding a disk to a virtual machine and the new disk does not show up:
One Liner Method
echo "- - -" > /sys/class/scsi_host/$(grep mpt /sys/class/scsi_host/host?/proc_name | grep -o -E '(host[0-9])')/scan
Manual Method
- Find the host bus number
grep mpt /sys/class/scsi_host/host?/proc_name
- Example return value:
/sys/class/scsi_host/host2/proc_name:mptspi
- “host2” is the important value
- Used the returned host# to scan the SCSI bus
echo "- - -" > /sys/class/scsi_host/host#/scan
- Example from returned value
echo "- - -" > /sys/class/scsi_host/host2/scan
- The hyphens represent controller,channel,lun, so – - – indicates all controllers, all channels, and all luns should be scanned.
Expand Disk: Re-Scan SCSI Bus
If you have expanded an existing disk, rescan the scsi bus
echo 1 > /sys/class/scsi_device/<device>/rescan
Where “<device>” is your disk such as “sda”.
Format the disk
/dev/sdb is used in this example.
Use gdisk in order for GPT instead of MBR. (seriously, don't use fdisk…let MBR die already)
gdisk /dev/sdb
Create a New Empty GUID Partition Table
Once in gdisk:
- Use option “o” to create a new empty GUID Partition table. (GPT)
- “w” to write table entries
Create the LVM Partition
- Go back into gdisk. (gdisk /dev/sdb)
- “n” to create a new partition
- Choose the defaults for partition number, start, and end blocks to fill the entire disk
- Use code “8e00” for Linux LVM
- “w” to write changes and exit
Create a Physical Volume
About: Think of physical volumes as your “virtual disks” to use in LVM.
Create a physical volume using the parition that the LVM is on.
pvcreate <device1-path> [device2-path] [deviceX-path]
Example:
pvcreate /dev/sdb1
Continue on to either Expanding Storage or Creating New Storage.
Expanding Storage
Expanding storage steps.
Expand: Summarized Steps
To expand storage for a logical volume, the summarized steps are:
Performed above in “Expanding or New”:
- Add virtual disk to VM
- Re-Scan SCSI Bus (if needed)
- Format the Disk with a partition
- Create physical volume using the new disk's partition
Remaining Steps:
- Extend the volume group using the new physical volume
- Extend the logical volume
- Verify new LVM layout
- Grow the file system
- Cleanup (if a pvmove of the logical volume was done to a new disk)
- Verify old physical volume is not used
- Reduce the volume group by removing the physical volume
- Remove the physical volume LVM label
- Delete virtual disk from virtual management UI
Extend the Volume Group
- Add the new physical disk to the existing volume group
vgextend vglocal /dev/sdb1
Extend Logical Volume
- Extend the logical volume that needs the space (/home will be used in this example)
- Option 1(Preferred): Move the physical extents of the logical volume to the new disk, so all of it resides on the same VMDK, and then use all of that new disk's space.
- There may be a performance hit if a logical volume is spanned across multiple VMDKs, so this method is preferred.
pvmove --name lvhome /dev/sda2 /dev/sdb1 lvextend --resizefs /dev/vglocal/lvhome /dev/sdb1
- Option 2: Give the logical volume all of the space from the newly added physical volume
lvextend --resizefs --extents +100%PVS /dev/vglocal/lvhome /dev/sdb1
- Option 3: Give the logical volume a specific amount of additional free space from a specific physical volume
lvextend --resizefs --size +10G /dev/vglocal/lvhome /dev/sdb1
- Option 4: Give the logical volume all of the free space available to the volume group (potentially across multiple physical volumes)
lvextend --resizefs --extents +100%FREE /dev/vglocal/lvhome
Extending a Logical Swap Volume
Extending a swap volume is slightly different, as the “–resizefs” flag will not work.
- Example: Specify the total size of the logical swap volume
lvextend --size 8G /dev/vglocal/lvswap /dev/sdb1
- Disable all swap
swapoff -a
- Create a new swap area, overwriting the original
mkswap /dev/vglocal/lvswap
- Enable swap
swapon -a
- Verify
swapon -s
Verify New LVM Layout
- Verify LVM Allocation
- Logical Volume:
lvs
- Volume Group:
vgs
- Physical Volumes:
pvs
Logical Volume Location(s) on Physical Volumes
If you would like to see what logical volumes reside where, a nice command to view physical extent to logical volume mappings is:
pvdisplay -m
Grow File System
Skip if you used the “–resizefs” flag to lvextend above.
- Grow the file system
- Ext2/3/4
resize2fs /dev/vglocal/lvhome
- XFS
xfs_growfs /dev/vglocal/lvhome
- Verify filesystem space
df -h
Cleanup
If you have just moved all of the data of a logical volume off of an old physical volume, the old physical volume can go away if it is no longer in use.
Examples below: You have just moved everything off of /dev/sdc1 to a new disk and can now remove the old disk (/dev/sdc).
Verify Old Physical Volume Not Used
pvs -o +pv_used
Reduce Volume Group
vgreduce vgname olddisk
vgreduce vglocal /dev/sdc1
Remove Physical Volume
pvremove olddisk
pvremove /dev/sdc1
Delete Virtual Disk
Login to the virtualization user interface and delete the old disk from inventory.
Errors During LVM Commands
- If you see errors during LVM commands like this after a device delete from vmware
root@llsrlscd01 home $ pvs /dev/sdc: read failed after 0 of 4096 at 0: Input/output error /dev/sdc: read failed after 0 of 4096 at 17179803648: Input/output error /dev/sdc: read failed after 0 of 4096 at 17179860992: Input/output error /dev/sdc: read failed after 0 of 4096 at 4096: Input/output error /dev/sdc1: read failed after 0 of 4096 at 17178755072: Input/output error /dev/sdc1: read failed after 0 of 4096 at 17178812416: Input/output error /dev/sdc1: read failed after 0 of 4096 at 0: Input/output error /dev/sdc1: read failed after 0 of 4096 at 4096: Input/output error PV VG Fmt Attr PSize PFree /dev/sda2 vglocal lvm2 a-- 19.50g 3.50g /dev/sdd1 vglocal lvm2 a-- 250.00g 0 /dev/sde1 vglocal lvm2 a-- 40.00g 0
- Fix by rebooting the system or
echo 1 > /sys/block/sdX/device/delete
- Where “sdX” is the device, such as 'sdc' in the above example.
Creating New Storage
Create New: Summarized Steps
To create new storage for a logical volume, the summarized steps are:
Performed above in “Expanding or New”:
- Add virtual disk to VM
- Re-Scan SCSI Bus (if needed)
- Format the Disk with a partition
- Create physical volume using the new disk's partition
Remaining Steps:
- Create a volume group
- Create a logical volume
- Create a file system on the logical volume
- Mount the logical volume
Create a Volume Group
About: Volume groups aggregate physical volumes into a usable pool of disk space.
vgcreate <vgname> <physical-device>
Example:
vgcreate vgstorage /dev/sdb1
Create a Logical Volume
About: Logical volumes use the disk space made available via a volume group.
lvcreate –size <size> –name <lv-name> <vg-name>
Example: Create a 100G logical volume
lvcreate --size 100G --name lvbackups vgstorage
Example: Create a logical volume that uses 100% of the volume group free extents
lvcreate --extents 100%FREE --name lvbackups vgstorage
Create the file system on the logical volume
About: So far, the only things being created were pretty much containers. Create the actual file system below.
mkfs -t <fstype> -L <label> /dev/mapper/<vgname>-<lvname>
Example:
mkfs -t ext4 -L Backups /dev/mapper/vgstorage-lvbackups
Mount the logical volume by uuid
Find the UUID with blkid
blkid /dev/mapper/vgstorage-lvbackups: UUID="d89744ad-133c-452b-98d8-9480ef18fe77" TYPE="ext4"
Add entry to /etc/fstab by UUID
vim /etc/fstab UUID=d89744ad-133c-452b-98d8-9480ef18fe77 /opt/backups ext4 defaults 0 2
Mount the fstab entries and check to see if it mounted with df
mount -a df -h
LVM Renaming
We don't always get a name right the first time. Luckily, LVM allows for easy renaming of logical volumes and volume groups.
Rename a Volume Group (vgrename vg-oldname vg-newname)
vgrename vgbackupmirror vgredusb
Rename a Logical Volume (lvrename vgname lv-oldname lv-newname)
lvrename vgredusb lvbackupmirror lvredbackups
LVM Snapshots
LVM allows you to “freeze” the metadata of files. This allows for things like rsync to complete successfully even if files it is trying to sync are modified during the operation.
{TODO}