PowerHA and EMC Busy Devices
While migrating data from an old EMC storage array to the new EMC All Flash array, we found that some of the EMC devices were giving an error that the specified device was busy when attempting to rmdev the device.
While the version of PowerHA is old now, I thought I would still blog this in case anyone has a similar issue.
I found another reference to this issue here: Powerha-holds-my-disks
EMC Device Details
-------------------------------------------------------------------------
DEVICE :VEND :PROD :REV :SER NUM :CAP(kb)
-------------------------------------------------------------------------
/dev/rhdisk12 :EMC :SYMMETRIX :5876 :24059ed000 : 11784960
node4:/# lsdev -Cc disk | grep hdisk12
hdisk12 Available 31-T1-01 EMC Symmetrix FCP MPIO VRAID
No volume group on the device.
node4:/# lspv | grep -w -f /tmp/Disk.List.txt
hdisk12 00abc7a7abc2dabd None
Current Software Levels.
node4:/# oslevel -s
7100-05-01-1731
node4:/# halevel -s
7.1.3 SP7
Removing the device gives an error.
node4:/# rmdev -l hdisk12
Method error (/etc/methods/ucfgdevice):
0514-062 Cannot perform the requested function because the
specified device is busy.
The device is not in a Fence Group and we appear to have read/write access.
node4:/# /usr/es/sbin/cluster/cspoc/cl_getdisk hdisk12
Disk name: hdisk12
Disk UUID: f671eedf241cc9cd 0265e4f89c1f3fef
Fence Group UUID: 0000000000000000 0000000000000000 - Not in a Fence Group
Disk device major/minor number: 19, 101
Fence height: 0 (Read/Write)
Reserve mode: 0 (No Reserve)
Disk Type: 0x01 (Local access only)
Disk State: 17
To fix this, I needed to take the cluster offline and then back online, just on this node. Note that I used the unmanaged option to avoid an application outage.
node4:/# clmgr offline node node4 when=now manage=unmanage broadcast=false
node4:/# clmgr online node node4 when=now manage=auto broadcast=false
Once the cluster software is up and running, I can delete the device.
node4:/# rmdev -dl hdisk12
hdisk12
hdisk12 deleted