The following is a brief overview of the process for adding LUNs to VXVM under Linux. In our example we have an RHEL 5 server with existing LUNs and VXVM volume groups. Two new LUNs with multipathing were allocated from SAN and need to be added to the system to grow one of the volumes and the corresponding filesystem. The first step is to make sure your system can see the new LUNs. A reboot is usually required for the OS to detect the new LUNs.

1. Run “vxdisk list” to see the list of disks detected by VXVM

# vxdisk list
DEVICE       TYPE            DISK           GROUP        STATUS
cciss/c0d0   auto:none       -              -            online invalid
san77721_0   auto:simple     sys-data-fs01  sys-data-fs  online thinrclm
san77721_1   auto:simple     sys-data-fs02  sys-data-fs  online thinrclm
san77721_2   auto:simple     sys-data-fs03  sys-data-fs  online thinrclm
san77721_3   auto:simple     sys-data-ft01  sys-data-ft  online thinrclm
san77721_4   auto:none       -              -            online invalid
san77721_5   auto:none       -              -            online invalid

In our case, disks san77721_4 and san77721_5 are the new LUNs. They do not yet belong to any disk groups.

2. Now you need to identify how Linux sees these two LUNs. First, use fdisk to get a list of all available disks:

# fdisk -l 2>/dev/null | egrep '^Disk' | egrep -v 'dm-'
Disk /dev/cciss/c0d0: 146.7 GB, 146778685440 bytes
Disk /dev/sda: 8589.9 GB, 8589934592000 bytes
Disk /dev/sdb: 8589.9 GB, 8589934592000 bytes
Disk /dev/sdc: 8589.9 GB, 8589934592000 bytes
Disk /dev/sdd: 4398.0 GB, 4398046511104 bytes
Disk /dev/sde: 4398.0 GB, 4398046511104 bytes
Disk /dev/sdf: 4398.0 GB, 4398046511104 bytes
Disk /dev/sdg: 8589.9 GB, 8589934592000 bytes
Disk /dev/sdh: 8589.9 GB, 8589934592000 bytes
Disk /dev/sdi: 8589.9 GB, 8589934592000 bytes
Disk /dev/sdj: 4398.0 GB, 4398046511104 bytes
Disk /dev/sdk: 4398.0 GB, 4398046511104 bytes
Disk /dev/sdl: 4398.0 GB, 4398046511104 bytes

Now you need to figure out which /dev/sd* device corresponds to which LUNs. It is important to keep in mind that, since we are dealing with multipathing, each LUN will be represented by two or more /dev/sd* devices. So the next step is to determine how many paths are used by each LUN:

fdisk -l 2>/dev/null | egrep '^Disk' | egrep -v 'dm-' | awk '{print $2}' | sed 's@/dev/@@g' | awk -F':' '{print $1}' | while read line
do
     echo ${line}
     vxdmpadm getdmpnode nodename=${line}
     echo ""
done

cciss/c0d0
NAME                 STATE        ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
==============================================================================
cciss/c0d0           ENABLED      OTHER_DISKS  1      1     0     other_disks

sda
NAME                 STATE        ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
==============================================================================
san77721_0           ENABLED      FAS6080      2      2     0     san77721

sdb
NAME                 STATE        ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
==============================================================================
san77721_1           ENABLED      FAS6080      2      2     0     san77721

sdc
NAME                 STATE        ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
==============================================================================
san77721_2           ENABLED      FAS6080      2      2     0     san77721

sdd
NAME                 STATE        ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
==============================================================================
san77721_3           ENABLED      FAS6080      2      2     0     san77721

sde
NAME                 STATE        ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
==============================================================================
san77721_4           ENABLED      FAS6080      2      2     0     san77721

sdf
NAME                 STATE        ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
==============================================================================
san77721_5           ENABLED      FAS6080      2      2     0     san77721

sdg
NAME                 STATE        ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
==============================================================================
san77721_0           ENABLED      FAS6080      2      2     0     san77721

sdh
NAME                 STATE        ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
==============================================================================
san77721_1           ENABLED      FAS6080      2      2     0     san77721

sdi
NAME                 STATE        ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
==============================================================================
san77721_2           ENABLED      FAS6080      2      2     0     san77721

sdj
NAME                 STATE        ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
==============================================================================
san77721_3           ENABLED      FAS6080      2      2     0     san77721

sdk
NAME                 STATE        ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
==============================================================================
san77721_4           ENABLED      FAS6080      2      2     0     san77721

sdl
NAME                 STATE        ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
==============================================================================
san77721_5           ENABLED      FAS6080      2      2     0     san77721

After reviewing this list, you will notice that san77721_4 corresponds to sde and sdk, while san77721_5 corresponds to sdf and sdl. This means that each LUN uses two paths. You can confirm this by running the vxdisk command for the specific disk device:

# vxdisk list sde | grep numpaths
numpaths:   2

3. Now we have determined that disks sde, sdk, sdf, and sdl are our two new LUNs. What we need to do now is to label and partition them. Labeling is done using “parted” command:

for i in sde sdk sdf sdl
do
     parted /dev/${i} mklabel gpt
done

Now we will use fdisk to partition the new disk devices:

fdisk /dev/sde
Select "p" to partition
Select "w" to write the partition table to disk

Repeat the last fdisk step for the remaining new disks.

4. After labeling and partitioning the new disk devices, we are ready to set up them in VXVM. To accomplish this, we will need to use the vxdisksetup command:

vxdisksetup -i san77721_4 format=simple
vxdisksetup -i san77721_5 format=simple

5. The next step is to add these two disks to the desired disk group (sys-data-fs, in our example) using vxdiskadd command:

vxdiskadd san77721_4

Add or initialize disks
Menu:: VolumeManager/Disk/AddDisks

  Here is the disk selected.  Output format: [Device_Name]

  san77721_4

Continue operation? [y,n,q,?] (default: y)

  You can choose to add this disk to an existing disk group, a
  new disk group, or leave the disk available for use by future
  add or replacement operations.  To create a new disk group,
  select a disk group name that does not yet exist.  To leave
  the disk available for future use, specify a disk group name
  of "none".

Which disk group [,none,list,q,?] sys-data-fs

Use a default disk name for the disk? [y,n,q,?] (default: y)

Add disk as a spare disk for sys-data-fs? [y,n,q,?] (default: n)

Exclude disk from hot-relocation use? [y,n,q,?] (default: n)

Add site tag to disk? [y,n,q,?] (default: n)

  The selected disks will be added to the disk group sys-data-fs with
  default disk names.

  san77721_4

Continue with operation? [y,n,q,?] (default: y)

  The following disk device appears to have been initialized already.
  The disk is currently available as a replacement disk.
  Output format: [Device_Name]

  san77721_4

Use this device? [y,n,q,?] (default: y)

  The following disk you selected for use appears to already have
  been initialized for the Volume Manager.
  Please use vxdisk -f init if you want to reinitialize this disk.


  VxVM  NOTICE V-5-2-88
Adding disk device san77721_4 to disk group sys-data-fs with disk
  name sys-data-fs04.

Goodbye.

We will repeat this process for the other LUN (san77721_5).

6. The two new LUNs are now a part of the disk group and can be used to grow the VXVM volume. To identify the volume corresponding to the disk group (sys-data-fs, in our example), use the vxprint command:

# vxprint -g sys-data-fs
TY NAME         ASSOC        KSTATE   LENGTH   PLOFFS   STATE    TUTIL0  PUTIL0
dg sys-data-fs  sys-data-fs  -        -        -        -        -       -

dm sys-data-fs01 san77721_0  -        16777143552 -     -        -       -
dm sys-data-fs02 san77721_1  -        16777143552 -     -        -       -
dm sys-data-fs03 san77721_2  -        16777143552 -     -        -       -
dm sys-data-fs04 san77721_4  -        8589857280 -      -        -       -
dm sys-data-fs05 san77721_5  -        8589857280 -      -        -       -

v  fs-vol       fsgen        ENABLED  50331428864 -     ACTIVE   -       -
pl fs-vol-01    fs-vol       ENABLED  50331428864 -     ACTIVE   -       -
sd sys-data-fs01-01 fs-vol-01 ENABLED 16777143552 0     -        -       -
sd sys-data-fs02-01 fs-vol-01 ENABLED 16777143552 16777143552 -  -       -
sd sys-data-fs03-01 fs-vol-01 ENABLED 16777141760 33554287104 -  -       -

In our case, the volume we are interested in is fs-vol. We can use the vxassist command to help us figure out just how big we can make this volume using all available disks in the associated disk group:

# vxassist -b maxgrow fs-vol -g sys-data-fs
Volume fs-vol can be extended by 17179715584 to: 67511144448 (32964426Mb)

So we can grow the volume up to 32Tb and that’s what we are going to do using the vxresize command:

# vxresize -g sys-data-fs fs-vol 67511144448

The volume resizing process may take a few minutes, depending on the size of the volume and the speed of storage and network. After giving the system a couple minutes to complete the operation, we can check on the size of the filesystem:

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/rootvg-rootlv
                       98G   15G   78G  16% /
/dev/cciss/c0d0p1      99M   20M   74M  22% /boot
tmpfs                  36G     0   36G   0% /dev/shm
/dev/vx/dsk/sys-data-fs/fs-vol
                       32T   21T   11T  66% /sdfs_lux01
/dev/vx/dsk/sys-data-ft/ft-vol
                      4.0T  154G  3.9T   4% /sdft_lux01

The fs-vol and the associated filesystem are now 32Tb. We hope this brief tutorial was helpful to you. Always remember to backup your data before making any changes to your storage. Operate under the assumption that if anything can go wrong, it will go wrong.