I got in trouble when I tried to remove a multipath device from my servers. This device is on top on some lvm volumes that I am not using it any longer. I tried to remove with multipath -f, but it was not possible, it said that the map was in use:
~# multipath -f /dev/mapper/2554b454e79496758 Dec 05 12:22:31 | 2554b454e79496758: map in use Dec 05 12:22:31 | failed to remove multipath map 2554b454e79496758
You can view how many processes are using this map with the dmsetup tool, see the open count field:
~# dmsetup info /dev/mapper/2554b454e79496758 Name: 2554b454e79496758 State: ACTIVE Read Ahead: 256 Tables present: LIVE Open count: 4 Event number: 1086846 Major, minor: 251, 4 Number of targets: 1 UUID: mpath-2554b454e79496758
First remove any LVM active devices on this device, ex: if you have a volume group “vggroup” and a lvm “vol1” on this device remove them:
~# lvremove /dev/vggroup/vol1 ~# vgremove vggroup ~# pvremove /dev/mapper/2554b454e79496758
and if the device file is still mapped under /dev remove it:
~# dmsetup remove /dev/vggroup/*
At this point there shouldn’t be any processes accessing this device and we should be able to remove it with the command above, but it some cases there are still processes blocked waiting for the device. We can try to find out which processes are with lsoft command filtering by device mayor and minor number:
~# lsof | grep "251,4"
In my case there was some vgs processes blocked trying to access the device. We cannot kill these processes, because they are already waiting for a signal from the kernel.
~# ps aux | grep sbin/vgs root 1206972 0.0 0.0 32444 4288 ? D dic02 0:00 /sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free root 1213321 0.0 0.0 32444 4308 ? D dic02 0:00 /sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free root 1248170 0.0 0.0 32444 4196 ? D dic02 0:00 /sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free root 2542017 0.0 0.0 32444 4252 ? D 10:46 0:00 /sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free
We can try to suspend the multipath device to force timeout to the processes:
~# dmsetup suspend /dev/mapper/2554b454e79496758 ~# dmsetup info /dev/mapper/2554b454e79496758 Name: 2554b454e79496758 State: SUSPENDED Read Ahead: 256 Tables present: LIVE Open count: 4 Event number: 1086846 Major, minor: 251, 4 Number of targets: 1 UUID: mpath-2554b454e79496758
And try to clear the device table:
~# dmsetup clear /dev/mapper/2554b454e79496758 ~# dmsetup wipe_table /dev/mapper/2554b454e79496758
We are lucky and finally the device is not in use any longer:
~# dmsetup info /dev/mapper/2554b454e79496758 Name: 2554b454e79496758 State: ACTIVE Read Ahead: 256 Tables present: LIVE Open count: 0 Event number: 1086846 Major, minor: 251, 7 Number of targets: 1 UUID: mpath-2554b454e79496758
Now, we can remove it without problems:
~# mutipath -f /dev/mapper/2554b454e79496758
To avoid multipath rediscover the device again we can blacklist it. Remove device from already discovered devices:
~# sed -i '/2554b454e79496758/d' /etc/multipath/wwids
In the multipath configuration file add an entry in the blacklist section withe the wwid of the deivce, if the file does not exist create it:
/etc/multipath.conf
blacklist {
wwid 2554b454e79496758
}
And finally reload multipath:
~# systemctl reload multipath-tools