Solaris SVM mirroring root disks with Solaris volume manager

Solaris SVM mirroring root / boot disks with Solaris volume manager

APPLIES TO :  Oracle Solaris 8, 9, 10

ISSUE :  NIL

GOAL :  Mirror the Solaris root disk

SOLUTION :  Find below steps to mirror the root disk of Solaris 10 server.  

Prerequisites :   NIL


               1. Copy Partition table from root to mirror disk..
                       prtvtoc /dev/rdsk/c0t0d0s2 | fmthard -s - /dev/rdsk/c0t0d1s2

               2. Create the metadb state databases
                       metadb -afc 3 /dev/dsk/c0t0d0s7 /dev/dsk/c0t0d1s7

               3. To see the new state database replicas we just created :
                       metadb

               4. Create two submirrors for the root volume
                       metainit -f d11 1 1 c0t0d0s0
                       metainit d12 1 1 c0t0d1s0

                       metainit -f d21 1 1 c0t0d0s1
                       metainit d22 1 1 c0t0d1s1

                       metainit -f d31 1 1 c0t0d0s3
                       metainit d32 1 1 c0t0d1s3

                       metainit -f d41 1 1 c0t0d0s4
                       metainit d42 1 1 c0t0d1s4

                       metainit -f d51 1 1 c0t0d0s5
                       metainit d52 1 1 c0t0d1s5

                       metainit -f d61 1 1 c0t0d0s6
                       metainit d62 1 1 c0t0d1s6

               5. Setup one way mirror OR create main mirror with primary disk's sub mirror
                       metainit -f d100 -m d11
                       metainit d200 -m d21
                       metainit d300 -m d31
                       metainit d400 -m d41
                       metainit d500 -m d51

                       metainit d600 -m d61

               6. Backup the configuration files
                       cp /etc/vfstab /etc/vfstab.premirr
                       cp /etc/system /etc/system.premirr

               7. Activiate the Mirror using the metaroot command which will update the /etc/system and /etc/vfstab files
                       metaroot d100
               8. Verify the configuration  file /etc/system and /etc/vfstab
                       cat /etc/system|grep "md"
                              rootdev:/pseudo/md@0:0,10,blk --> This extra line reflects the mirror configuration


                       cat /etc/vfstab|grep -i md
              dev/md/dsk/d100 /dev/md/rdsk/d100        /       ufs     1    --> the root volume entry in vfstab modified.                                        

               9. Modify the system file
                       echo "set md:mirrored_root_flag=1" >> /etc/system

               10. Modify the vfstab for other metadevice entries
                       /dev/md/dsk/d200 - - swap - no -
                       /dev/md/dsk/d100 /dev/md/rdsk/d100 / ufs 1 no -
                       /dev/md/dsk/d400 /dev/md/rdsk/d400 /usr ufs 1 no -
                       /dev/md/dsk/d300 /dev/md/rdsk/d300 /var ufs 1 no -
                       /dev/md/dsk/d500 /dev/md/rdsk/d500       /opt    ufs     2       yes     -
                       /dev/md/dsk/d600 /dev/md/rdsk/d600 /oracle ufs 2 yes logging

               11. Before rebooting, run the sync command multiple times and lockfs -fa command to flush the data to file system
                       sync
                       sync
                       sync

                       lockfs -fa

               12. Now we can reboot the system.
                       shutdown -i6 -g0 -y

               13. Attaching the other submirror to the main mirror and sync the data
                       metattach d100 d12
                       metattach d200 d22
                       metattach d300 d32
                       metattach d400 d42
                       metattach d500 d52
                       metattach d600 d62

               14. Change the swap partition from slice device to meta device
                       swap -l
                       swap -d /dev/dsk/c0t0d0s1
                       swap -a /dev/md/dsk/d0

               15. if the dump device shows any disk slice path change it to meta device path.
                       dumpadm
                       dumpadm -d /dev/md/dsk/d1

               16. The sync will start now. To check the status of the mirror 
                       metastat -cp

               17. Install the boot block on the mirror disk
                       installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c0t0d1s0

               18. Reboot the server with both primary and secondary disks to check the stability of the mirror functionality. 



~ Judi ~

Solaris Volume Manager (SVM) Command Line Reference

Solaris Volume Manager (SVM) Command Line Reference

Commands and Configuration files location:
For Solaris Volume Manager Solaris 8, 9, 10:
           Commands are  in /usr/sbin/
           Configuration files are in /etc/lvm/

Configuration files used with metadb.
           SDS:   /etc/system, /etc/opt/SUNWmd/mddb.cf
           SVM:  /etc/lvm/mddb.cf /kernel/drv/md.conf

The md.tab file Located in /etc/lvm/md.tab (SVM) and /etc/opt/SUNWmd/md.tab (SDS).  The file may be used to automatically create metadevices.

The file is empty (by default)
You may populate the file by appending the output of # metastat -p. For example #metastat -p >> /etc/lvm/md.tab
The md.tab file is never used unless the administrator issues a metainit command to read it.
The most common usage is #metainit -a (create all metadevices in md.tab), and #metainit dxx (create metadevice dxx only)]
Best used in recovery of SVM configurations .
Not recommended to be used on the root file system.

Local Database Replicas aka local metadb:
Put 3 replicas on c0t0d0s7.
           # metadb -a -f -c 3 c0t0d0s7

Create 3 more copies each on two disk drive slices.
           # metadb -a -c 3 c0t1d0s7
           # metadb -a -c 3 c0t2d0s7

Deleting Replicas.
           # metadb -d c0t1d0s7

Deleting your last replica (You SVM configuration will be gone.)
           # metadb -d -f c0t0d0s7

Checking meta database status:
           # metadb -i

Creating a Concatenation:

Creating a Concatenation from slice 2 of 3 disk drives:
           # metainit d1 3 1 c0t1d0s2 1 c1t1d0s2 1 c2t1d0s2
           d1 - the metadevice
           3   - the number of components to concatenate together
           1   - the number of devices for each component.

Creating a Simple Stripe from slice 2 of 3 disk drives.
           # metainit d2 1 3 c0t1d0s2 c1t1d0s2 c2t1d0s2 -i 16k
           d2      - the  metadevice
           1        - the number of components to to concatenate
           3        - the number of devices in each stripe.
           -i 16k - the stripe segment size

A more complicated example. 3, "two disk concatenations"  are striped together.
           # metainit d3 3 2 c0t1d0s2 c1t1d0s2  -i 16k 2 c3t1d0s2 c4t1d0s2  -i 16k 2 c6t1d0s2 c7t1d0s2  -i 16k
           d3     - the meatadevice
           3       - the number of stripes
           2       - the number of disk (slices) in each stripe
           -i 16k - the stripe segment size.

Growing, extending a metadevice
           # metattach d1 c3t1d0s2
         
extends a metadevice by concatenating a slice to the end. It does not expand a filesystem. You have to grow UFS filesystem once the metadevice was extended.
           # growfs /dev/md/rdsk/d1
           If the metadevice is not mounted, the above command extends the filesystem to include the added section. You cannot shrink this filesystem later.
         
           # growfs -M /export/home /dev/md/rdsk/d1
           If the metadevice is mounted, the above command will extend the filesystem to include the concatenated section. Again, you cannot make the filesystem smaller later.

Removing a metadevice
           # metaclear d3
           d3 is the metadevice being removed.
           clears, deletes all metadevices. Don't do this unless you want to blow away your entire configuration.
           The devices cannot be open for use.., i.e. mounted

Viewing your configuration and status:
Shows the configuration and status of all metadevices         
           # metastat -p

Will tell the configuration and status of just metadevice d3
           # metastat d3
         
Tells the location and status of locally configured replicas
           # metadb
           Note that: these commands displays the configuration on the local filesystems and not on the disksets, metasets. For metasets add -s <setname>

Hot Spare pools:
Sets up a pool called hsp001. It contains no disks yet.
           # metainit hsp001
         
Adds a slice to the hot spare pool. 
           # metahs -a hsp001 c0t1d0s4

Adds a slice to all pools         
           # metahs -a all c1t1d0s4
         
Makes a hot spare pool available to the metadevice d1 {submirror or RAID5}
           # metaparam -h hsp001 d1
         
Reenables a hot spare that was previously unavailable
           # metahs -e c1t1d0s4

Replaces the first disk slice listed with the second
           # metahs -r hsp001 c1t1d0s4 c2t1d0s4

Removes a disk slice from all hot spare pools
           # metahs -d all c1t1d0s4

Removes a disk slice from hsp001
           # metahs -d hsp001 c1t1d0s4

Removes a hot spare pool
           # metahs -d hsp001

Reports the Disksuite/LVM status
           # metahs -i
           # metastat

Mirrors:

           # metainit d0 -m d1
           Makes a one-way mirror. d0 is the device to mount, but d1 is the only one associated with an actual device.
           A "one-way mirror" is not really a mirror yet. There's only one place where the data is actually
stored, namely d1.

Attaches d2 to the d0 mirror.
           # metattach d0 d2
           Now there are 2 places where the data are  stored, d1 and d2. But you mount the metadevice d0.

Detaches d1 from the d0 mirror
           # metadetach d0 d1

Suspends/resumes use of d2 device on d0 mirror
           # metaoffline d0 d2
           # metaonline d0 d2

Replaces first disk listed with second on the d0 mirror
           # metareplace d0 c1t0d0s2 c4t1d0s2

Re-enables a disk that has been errored.
           # metareplace -e d0 c1t1d0s2
           re-enables a disk that has been errored.


Mirroring root:
You must take a few extra steps to mirror the root partition
           # metainit -f d1 1 1 c0t3d0s0 <-- the root partition
           # metainit d0 -m d1
           # metaroot d0
           The metaroot command updates /etc/system and /etc/vfstab so that the device /dev/md/dsk/d0 is now the root device.
           Note: It is recommended to take a copy of the /etc/system and /etc/vfstab before running metaroot command or making any change in these files.

You must reboot with a one-way mirror : do not create a 2 way mirror before rebooting otherwise the system will crash because of the round robin manner in which data is read.
           # metainit d2 1 1 c0t4d0s0
           # metattach d0 d2
           Now d2 is attached and data is mirrored on d1 and d2.

Note : this procedure mirrors only /, if you want to mirror the whole system disk, do not forget to mirror the swap slice and all other slices where a file system is installed using the above procedure of creating submirrors and mirrors.
In this case, remember that metaroot command only modify / in vfstab, so you have to manually edit other system disk entries in /etc/vfstab to put metadevice paths before rebooting.

Raid 5:
Sets up a RAID 5 configuration.
           # metainit d1 -r c0t1d0s2 c1t1d0s2 c2t1d0s2 -i 16k
           The -i option is the same as in striping.

Replacing disks as in the mirror.
           # metareplace d1 c2t3d0s2 c3t1d0s2
           # metareplace -e d1 c0t1d0s2

Concatenates a disk to the end of the RAID 5 configuration.
           # metattach d1 c4t3d0s2

Adds a hot spare pool
           # metaparam -h hsp001 d1

Removes a metadevice
           # metaclear d1

Tells status
           # metastat



UFS logging:   (obsolete, UFS has this now by default)
Sets up a trans device d0 with d1 as the master and d2 as the logging device.
           # metainit d0 -t d1 d2
           Recomended 1MB logging/1GB data on master
Same as above
           # metainit d0 -t c0t1d0s2 c3t2d0s5

Attaching and detaching a log device on/from d0
           # metattach d0 d1
           # metattach d0 c3t1d0s5
           # metadetach d0

Disksets:
You can do almost everything the same way, except specify -s <deskset>
Disks are repartitioned when put into a diskset unless slice 2 is zeroed out and slice 7 has cylinders 0-4 or 0-5 allocated to it for the diskset metadb

           # command -s <setname> options

Adds hosts to a set
           # metaset -s <setname> -a -h <hostname1> <hostname2>

Adds drives to a set. Notice we do not specify slice
           # metaset -s <setname> -a c2t0d0 c2t1d0 c2t2d0 c2t3d0

Removes hosts and drives.
           # metaset -s <setname> -d c2t3d0
           # metaset -s <setname> -d -h <hostname>

Release control of a diskset:
           # metaset -s <setname> -r

View the errored metadevice alone
           # metastat | awk '/State:/ { if ( $2 != "Okay" ) if ( prev ~ /^d/ ) print prev, $0 } { prev = $0 }' 

View the medadevice status

           # metastat | awk '/State:/ {if ( prev ~ /^d/ ) print prev, $0 } { prev = $0 }' #Will get all Metadevice status


Take control of a diskset. the -f option will force control but will panic other machine, unless it has been released from other host.

Solaris Volume Manager (SVM): Best Practices for Creation and Implementation of Soft Partitions ( Doc ID 1417827.1 )
Solaris Volume Manager (SVM) Command Line Reference ( Doc ID 1011732.1 )
Analyzing Internal non-RAID Disk Failures for x64 Solaris ( Doc ID 1017472.1 )

Solaris 10 x86 SVM Patching

Solaris 10 x86 SVM Patching
Solaris 10 SVM Patching (x86)

Step 1:     Backup the necessary configuration file and save it
                          df -h
                          metastat -p
                          metadb
                          echo | format
                          prtconf -v | sed -n '/bootpath/{;p;n;p;}'
                  Root Disk - c0t0d0
                  Root Mirror Disk - c0t1d0
                  6 copies of metadb replicas - c0t0d0s7 and c0t1d0s7
                  Current Kernel - 147441-01

Step 2:      Detatch the submirrors and clear them
                  /dev/md/dsk/d0 /
                  /dev/md/dsk/d1 swap
                  /dev/md/dsk/d3 /var
                  /dev/md/dsk/d4 /opt/BMC

                   JUDI-DEV-TEST01# metastat -p
                   d4 -m d14 d24 1
                   d14 1 1 c0t0d0s4
                   d24 1 1 c0t1d0s4
                   d3 -m d13 d23 1
                   d13 1 1 c0t0d0s3
                   d23 1 1 c0t1d0s3
                   d1 -m d11 d21 1
                   d11 1 1 c0t0d0s1
                   d21 1 1 c0t1d0s1
                   d0 -m d10 d20 1
                   d10 1 1 c0t0d0s0
                   d20 1 1 c0t1d0s0
                   JUDI-DEV-TEST01#
                          metastat -p
                          metadetach d0 d20
                          metadetach d1 d21
                          metadetach d3 d23
                          metadetach d4 d24

                          metastat -p

                          metaclear d20
                          metaclear d21
                          metaclear d23
                          metaclear d24

                          metastat -p

Step 3:      Remove replicas added on root mirror disk
                          metadb
                          metadb -d c0t1d0s7
                          metadb

Step 4:      Install grub on root mirror disk to make sure the disk is bootable incase we want to back-out the patching
                          installgrub -m /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c0t1d0s0

Step 5:      Mount root Mirror disk c0t1d0s0 on mnt
                          mount /dev/dsk/c0t1d0s0 /mnt
                          df -h /mnt

Step 6:      Modify /mnt/etc/vfstab & /mnt/etc/system files
                          cat /mnt/etc/vfstab
                          vi /mnt/etc/vfstab
                          /dev/dsk/c0t1d0s1  -       -       swap    -       no      -
                          /dev/dsk/c0t1d0s0  /dev/rdsk/c0t1d0s0 /       ufs     1       no      -
                          /dev/dsk/c0t1d0s3  /dev/rdsk/c0t1d0s3 /var    ufs     1       no      -
                          /dev/dsk/c0t1d0s4  /dev/rdsk/c0t1d0s4 /opt/BMC        ufs     2       yes     -

                          tail /mnt/etc/system
                          vi /mnt/etc/vfstab (Comment the mirror related information)
                          * Begin MDD root info (do not edit)
                          * rootdev:/pseudo/md@0:0,0,blk
                          * End MDD root info (do not edit)
                          * set md:mirrored_root_flag=1

Step 7:      Delete existing boot sign on root mirror disk and create unique one
                          ls -l /mnt/boot/grub/bootsign/
                          -r--r--r--   1 root     root           0 Oct 17  2013 rootfs0

                          rm /mnt/boot/grub/bootsign/rootfs0
                          touch /mnt/boot/grub/bootsign/rootfs1
                          ls -l /mnt/boot/grub/bootsign/

Step 8:      Update root mirror disk menu.lst file
                          ls -lrt /mnt/boot/grub/menu.lst
                          cat /mnt/boot/grub/menu.lst

                  Edit the file and change the rootfs0 to rootfs1 in both lines and change the title as Root Mirror Disk
                          vi /mnt/boot/grub/menu.lst
                          #---------- ADDED BY BOOTADM - DO NOT EDIT ----------
                          title Root Mirror Disk Oracle Solaris 10 8/11 s10x_u10wos_17b X86
                          findroot (rootfs1,0,a)
                          kernel /platform/i86pc/multiboot
                          module /platform/i86pc/boot_archive
                          #---------------------END BOOTADM--------------------
                          #---------- ADDED BY BOOTADM - DO NOT EDIT ----------
                          title Solaris failsafe
                          findroot (rootfs1,0,a)
                          kernel /boot/multiboot -s
                          module /boot/amd64/x86.miniroot-safe
                          #---------------------END BOOTADM--------------------

                          cat /mnt/boot/grub/menu.lst

Step 9:      Update boot environment variable on root mirror disk
                          echo | format
                          cat /mnt/boot/solaris/bootenv.rc
                          ls -ld /dev/dsk/c0t1d0s0
                          lrwxrwxrwx   1 root     root          62 Oct 17  2013 /dev/dsk/c0t1d0s0 -> ../../devices/pci@0,0/pci8086,3c06@2,2/pci1028,1f38@0/sd@1,0:a
                          vi /mnt/boot/solaris/bootenv.rc
                          setprop bootpath '/pci@0,0/pci8086,3c06@2,2/pci1028,1f38@0/sd@0,0:a' ----> Remove this line
                          setprop bootpath '/pci@0,0/pci8086,3c06@2,2/pci1028,1f38@0/sd@1,0:a' ----> Add the secondary disks path here

Step 10:      Update boot disks menu.lst file,
                  This step will allow us to skip the step of configuring BIOS to boot from root mirror disk
                          cat /boot/grub/menu.lst
                  Edit the file and make entry to list out the secondary disk as separate boot disk while grub booting screen comes
Add the below entry to the bottom of the file - secondary disk - rootfs1
                          #---------- ADDED BY BOOTADM - DO NOT EDIT ----------
                          title Root Mirror Disk - Oracle Solaris 10 8/11 s10x_u10wos_17b X86
                          findroot (rootfs1,0,a)
                          kernel /platform/i86pc/multiboot
                          module /platform/i86pc/boot_archive
                          #---------------------END BOOTADM--------------------
                          #---------- ADDED BY BOOTADM - DO NOT EDIT ----------
                          title Solaris failsafe
                          findroot (rootfs1,0,a)
                          kernel /boot/multiboot -s
                          module /boot/amd64/x86.miniroot-safe
                          #---------------------END BOOTADM--------------------

Step 11:      Check the currently booted device
                          prtconf -v | sed -n '/bootpath/{;p;n;p;}'
                  Currently server booted from root disk, Now restart and boot from root mirror disk
                          init 6
                  Server will display as
                  creating boot_archivee for /mnt
                  updating /mnt/platform/i86pc/boot_archive

                  while rebooting slect the root mirror disk from grub menu - to check the root secodary disk is safe to boot.
                  Check from which disk the server is booted and make sure the disk booted from root mirror - now we are good to proceed patching
                          prtconf -v | sed -n '/bootpath/{;p;n;p;}'

Step 12:      Reboot the server to boot from boot disk see the root disk status
                          init 6
                  Check from which disk the server is booted and make sure the disk booted from boot disk - Primary disk
                          prtconf -v | sed -n '/bootpath/{;p;n;p;}'

Step 13:      Bring the server into single user mode and install the patches
                          who -r
                          init s

Step 14:      Start install the patches
                          ./installpatchset --s10patchset

Step 15:      Patch installation completed, Reboot the server
                          init 6
                  Verify the new patch version and smf status
                          uname -v
                          svcs -xv
                          df -h

Step 16:      Now create submirrors and replicas on root mirror disk
                          echo | format
                          metstat -p
                          metainit -f d0 1 1 c0t1d0s0
                          metainit -f 21 1 1 c0t1d0s1
                          metainit -f 23 1 1 c0t1d0s3
                          metainit -f 24 1 1 c0t1d0s4

Step 17:      Attach submirrors to the main mirror
                          metadb -afc3 c2t0d0s7
                          metadb
                          metstat -c

                          metattach d0 d0
                          metattach d1 d21
                          metattach d3 d23
                          metattach d4 d24

                          metastat -c
                  Wait until the resync completes

                          uname -X
                          metastat -c d0

                  Once the resync completed, Reboot the server
                          init 6



Solaris10
Solaris 10 x86 SVM Patching
Solaris 10 SVM Patching (x86);
how to patch solaris 10 x86 server;
Update Solaris 10 kernel version;
Solaris 10 x86 Kernel patching;

~Judi~