VxVM : How to split a mirror in Veritas Volume manager

This document describes how to split a Veritas mirror so that the sub-mirrors can be associated with new volumes, and hence be accessed via Solaris[TM] devices and possibly mounted.

Be aware that you may not be able to mount a filesystem taken from a split mirror, depending on the state of the filesystem at the time of the split. If a filesystem is unmounted cleanly prior to the split, both copies should be mountable afterwards.

So this is our starting point, a single volume with two plexes forming a simple mirror in the disk group 'baddg'.

# vxprint -g baddg
TY NAME         ASSOC        KSTATE   LENGTH   PLOFFS   STATE    TUTIL0  PUTIL0
dg baddg        baddg        -        -        -        -        -       -

dm sd01         Disk_1       -        35358848 -        -        -       -

v  vol1         gen          ENABLED  207328   -        ACTIVE   -       -
pl plex1        vol1         ENABLED  207328   -        ACTIVE   -       -
sd sd01-B0      plex1        ENABLED  1        0        -        -       Block0
sd sd01-02      plex1        ENABLED  207327   1        -        -       -
pl plex2        vol1         ENABLED  207328   -        ACTIVE   -       -
sd sd01-01      plex2        ENABLED  207328   0        -        -       -

It has a filesystem which is mounted.

# mount |grep baddg
/a on /dev/vx/dsk/baddg/vol1
 read/write/setuid/intr/largefiles/logging/xattr/onerror=panic/dev=3b41f40
 on Thu Jul 21 16:05:26 2005

We write a file and calculate the checksum. (Note: This is not a necessary part of the procedure but will be used later in this document to illustrate that the procedure has been a success.)

# cp /kernel/genunix /a
# ls -l /a
total 3600
-rwxr-xr-x   1 root     other    1823432 Jul 21 16:08 genunix
drwx------   2 root     root        8192 Jul 21 16:02 lost+found
# cksum /a/genunix
411313929       1823432 /a/genunix

Now unmount the filesystem, so that we know everything has been flushed to the physical disk.

# umount /a

In the following steps we are going to disassociate the plexes from the parent volume using vxplex 'dis'. VxVM also has a vxplex 'det' which is to detatch a plex. Confusingly, detatching a plex with vxplex 'det', stops writes to the plex, but the plex stays associated with the parent volume.

The first step is to detach one of the plexes and break its link to the remaining plex(es) (i.e. dissociate). We'll dissociate 'plex2'

# vxplex -g baddg dis plex2

Let's check how VxVM sees things now - plex2 is no longer associated with the volume.

# vxprint -g baddg
TY NAME         ASSOC        KSTATE   LENGTH   PLOFFS   STATE    TUTIL0  PUTIL0
dg baddg        baddg        -        -        -        -        -       -

dm sd01         Disk_1       -        35358848 -        -        -       -

pl plex2        -            DISABLED 207328   -        -        -       -
sd sd01-01      plex2        ENABLED  207328   0        -        -       -

v  vol1         gen          ENABLED  207328   -        ACTIVE   -       -
pl plex1        vol1         ENABLED  207328   -        ACTIVE   -       -
sd sd01-B0      plex1        ENABLED  1        0        -        -       Block0
sd sd01-02      plex1        ENABLED  207327   1        -        -       -

Now create a volume using the disassociated plex (plex2). The volume will be called 'vol2'

# vxmake -g baddg vol vol2 plex=plex2

vxprint shows the new volume, but the volume is disabled.
v  vol2         gen          DISABLED 207328   -        EMPTY    -       -
pl plex2        vol2         DISABLED 207328   -        EMPTY    -       -
sd sd01-01      plex2        ENABLED  207328   0        -        -       -

Next, we must start the new volume

# vxvol -g baddg start vol2

The new volume is now available to mount.

v  vol2         gen          ENABLED  207328   -        ACTIVE   -       -
pl plex2        vol2         ENABLED  207328   -        ACTIVE   -       -
sd sd01-01      plex2        ENABLED  207328   0        -        -       -

We can now mount the two volumes that were previously two halves of a single mirror and see that the same file is there, and has the same checksum - i.e. is a consistent copy.

First we mount the new volume

# mount /dev/vx/dsk/baddg/vol2 /b
# ls -l /b
total 3600
-rwxr-xr-x   1 root     other    1823432 Jul 21 16:08 genunix
drwx------   2 root     root        8192 Jul 21 16:02 lost+found
# cksum /b/genunix
411313929       1823432 /b/genunix

Now the original volume (no longer a mirror)

# mount /dev/vx/dsk/baddg/vol1 /a
# ls -l /a
total 3600
-rwxr-xr-x   1 root     other    1823432 Jul 21 16:08 genunix
drwx------   2 root     root        8192 Jul 21 16:02 lost+found
# cksum /a/genunix
411313929       1823432 /a/genunix

To recreate the original mirror, follow the above steps to disassociate the plex from the new volume. Then use vxplex 'att' to attach the plex to the original volume. The plex will be synchronised with the plex(es) already attached to the volume.

VxVM 4.0 : Alignment error with device creation commands.

When using device creation commands, such as vxmake, an error is returned:

VxVM vxmake ERROR: Length for subdisk  is not a multiple of the disk group alignment.

VxVM vxmake INFO: Subdisk length is <13422294> while disk group alignment is 16

This error is due to the diskgroup alignment size in vxvm 4.0. The alignment size for 4.0, by default is 16 blocks (8k). This size is set to accommodate Cross-Platform Data Sharing(CDS) capability. Unless specified otherwise, CDS is assumed when a new volume is created, thus the alignment size is set to 16 blocks. In earlier versions, the alignment size was set at 1 block, and was not visible.

This can create problems when using / moving devices within diskgroups.

Resolution

QOALQ6H Internal ID use only.

First, check if the 2 diskgroups have differing alignment sizes:

To compare 2 different diskgroups, use the following command:

vxdg list

Example: (also, notice the flag "cds")

Group:     newdg
dgid:      1106270153.72.lab1
import-id: 1024.71
flags:     cds    <-------------
version:   110
alignment: 8192 (bytes)
ssb:            on
detach-policy: global
copies:    nconfig=default nlog=default
config:    seqno=0.1042 permlen=1280 free=1275 templen=2 loglen=192
config disk Disk_4 copy 1 len=1280 state=clean online
log disk Disk_4 copy 1 len=192

Example:

Group:     newdg1
dgid:      1106276811.60.lab1
import-id: 1024.59
flags:   
version:   110
alignment: 512 (bytes)
ssb:            on
detach-policy: global
copies:    nconfig=default nlog=default
config:    seqno=0.1044 permlen=1280 free=1275 templen=2 loglen=192
config disk Disk_4 copy 1 len=1280 state=clean online
log disk Disk_4 copy 1 len=192

Here, we can see the 2 differing alignment fields, one is set at 8192 bytes, the other at 512 bytes, reflecting the 8 kB -vs- 1 block difference.

If not utilizing the CDS feature, you can set the CDS to off, then change the alignment size of the diskgroup. The CDS feature must first be disabled for the diskgroup, or it will not allow you to change the alignment size.

Example:

#vxdg -g  set cds=off
#vxdg -g  set alignment=1

After these commands, the diskgroups' alignment should now show as 512 in the vxdg list output.

Example:
Group:     newdg
dgid:      1106270153.72.lab1
import-id: 1024.71
flags:    
version:   110
alignment: 512 (bytes)
ssb:            on
detach-policy: global
copies:    nconfig=default nlog=default
config:    seqno=0.1042 permlen=1280 free=1275 templen=2 loglen=192
config disk Disk_4 copy 1 len=1280 state=clean online
log disk Disk_4 copy 1 len=192

Additional Information

QOVLUR4 Internal ID use only.

Example:

Create diskgroup, volumes, etc, in any earlier version of VxVM. (non-4.0)
Mirror volume.
Attach the mirror plex, allow to sync.
Detach one of the mirror plexes.
Upgrade to vxvm 4.0
Create new diskgroup, use pre-existing plex. 

Here is where we would see the error. Once we try to associate the existing plex with a new diskgroup, the alignment mismatch will appear.

This could come up in several different instances. Some customers use their own scripted method of a "snapshot" by adding a 3rd plex to a mirrored volume, syncing the plex, detaching & creating a new diskgroup using this plex. They would then use the plex to create a new volume & diskgroup as means of a backup.

Product

VERITAS Volume Manager 4.1 Software
VERITAS Volume Manager 4.0 Software
VERITAS Storage Foundation 4.1 Software
VERITAS Storage Foundation 4.0 Software

ringtones