Oracle ASMLib Kernel driver upgdrade (Standalone 11GR2)

I. As RDBMS and GI/ASM user:

1) Shutdown all Databases (as RDBMS user) utilizing the ASM instance(s)

Identify the database need to be shutdown that is clients of ASM instance.

SELECT instance_name, db_name FROM v$asm_client WHERE status = 'CONNECTED';

AS oracle User

# sudo su - oracle

# export ORACLE_SID=gridrep

# sqlplus / as sysdba

SQL> shutdown immdediate;

2) Shutdown all ASM instance(s)

AS ASM User

# sudo su - grid

#export ORACLE_SID= +ASM

# sqlplus / as sysdba

SQL> shutdown immdediate;

3) Stop all other related services like the Enterprise Manager Agent, etc., if being used.

I am doing this in OMS so stopping Agent and OMS:

Stop Agent:

ORACLE_CONFIG_HOME=/opt/oracle/Middleware/gc_inst/em/EMGC_OMS1

ORACLE_BASE=/opt/oracle

ORACLE_HOME=/opt/oracle/Middleware/agent/agent_inst

PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:/sbin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/oracle/bin

export ORACLE_BASE ORACLE_HOME ORACLE_CONFIG_DIR PATH

emctl stop agent

Stop OMS:

ORACLE_CONFIG_HOME=/opt/oracle/Middleware/gc_inst/em/EMGC_OMS1

ORACLE_BASE=/opt/oracle

ORACLE_HOME=/opt/oracle/Middleware/oms

PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:/sbin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/oracle/bin

export ORACLE_BASE ORACLE_HOME ORACLE_CONFIG_DIR PATH

emctl stop oms

II. As root user:

# /etc/init.d/oracleasm listdisks

GRIDREP_0001

# ls -l /dev/oracleasm/disks/

total 0

brw-rw---- 1 grid asmadmin 105, 1 Jan 22 16:18 GRIDREP_0001

# for i in `/etc/init.d/oracleasm listdisks`

do

v_asmdisk=`/etc/init.d/oracleasm querydisk -d $i | awk '{print $2}'`

v_minor=`/etc/init.d/oracleasm querydisk -d $i | awk -F[ '{print $2}'| awk -F] '{print $1}' | awk -F, '{print $1}'`

v_major=`/etc/init.d/oracleasm querydisk -d $i | awk -F[ '{print $2}'| awk -F] '{print $1}' | awk -F, '{print $2}'`

v_device=`ls -la /dev | grep $v_minor | grep $v_major | awk '{print $10}'`

echo "ASM disk $v_asmdisk based on /dev/$v_device [$v_minor $v_major]"

done

ASM disk "GRIDREP_0001" based on /dev/ [105 1]

Make a reference note of the ASM disk above which will be useful later for double check everything.

2) Make a backup copy of the file, /etc/sysconfig/oracleasm

a. # ls -la /etc/sysconfig/oracleasm

lrwxrwxrwx 1 root root 24 Mar 8 2012 /etc/sysconfig/oracleasm -> oracleasm-_dev_oracleasm

b. # cp /etc/sysconfig/oracleasm-_dev_oracleasm /etc/sysconfig/oracleasm-_dev_oracleasm.orig

3) Shutdown all services:

# /opt/grid/product/11.2.0.3/bin/crs_stat -t

Name Type Target State Host

------------------------------------------------------------

ora.DATA.dg ora....up.type ONLINE ONLINE dbms2

ora....ER.lsnr ora....er.type ONLINE ONLINE dbms2

ora.asm ora.asm.type ONLINE ONLINE dbms2

ora.cssd ora.cssd.type ONLINE ONLINE dbms2

ora.diskmon ora....on.type OFFLINE OFFLINE

ora.evmd ora.evm.type ONLINE ONLINE dbms2

ora.gridrep.db ora....se.type ONLINE ONLINE dbms2

ora.ons ora.ons.type OFFLINE OFFLINE

# /opt/grid/product/11.2.0.3/bin/crsctl stop resource -all

4) As root user:

a) Stop HAS:

# /opt/grid/product/11.2.0.3/bin/crsctl stop has

b) Check HA to ensure it is down:

#/opt/grid/product/11.2.0.3/bin/crsctl check has

c) Disable HA for auto startup:

#/opt/grid/product/11.2.0.3/bin/crsctl disable has

d) Check /etc/inittab and comment out respawn of ohasd.

1. # vi /etc/inittab

2. Comment out (#) the following line in /etc/inittab:

#h1:35:respawn:/etc/init.d/init.ohasd run >/dev/null 2>&1 </dev/null

3. # /sbin/init q

4. Ensure that init.ohasd is now not running

ps -ef|grep init

e) Stop CRS:

#/opt/grid/product/11.2.0.3/bin/crsctl stop crs

f) Check CSS and CRS to ensure it is down:

#/opt/grid/product/11.2.0.3/bin/crsctl check css

#/opt/grid/product/11.2.0.3/bin/crsctl crsctl check crs

III. Stop oracleasm (as root)

1) Stop oracleasm:

a) # /etc/init.d/oracleasm stop

Dropping Oracle ASMLib disks: [ OK ]

Shutting down the Oracle ASMLib driver: [ OK ]

b) # /etc/init.d/oracleasm status

Checking if ASM is loaded: no

Checking if /dev/oracleasm is mounted: no

OR

# service oracleasm status

Checking if ASM is loaded: no

Checking if /dev/oracleasm is mounted: no

2) Disable oracleasm for not starting on reboot:

# /etc/init.d/oracleasm disable

Writing Oracle ASM library driver configuration [ OK ]

Unmounting ASMlib driver filesystem [ OK ]

Unloading module "oracleasm" [ OK ]

IV. Upgrade OS Kernel (System Administrator will do this.):

1) Verify the new kernel version:

# rpm -q oracleasm-`uname -r`

oracleasm-2.6.18-128.1.1.0.1.el5xen-2.0.5-1.el5

The kernel module "oracleasm" is required to be an exact match to the running kernel version in order for ASMLib to operate.

2) Deinstall existing oracleasm driver packages (optional).

3) Download the correct ASMLib drivers and Install

Oracle Enterprise Linux users of ASMLib should download the same product version for the new, upgrade kernel using up2date/ULN (http://linux.oracle.com) only. For steps to upgrade OS Linux Karnel, please check Doc ID 842775.1

Red Hat Enterprise Linux users ASMLib should download the same product version for the new, upgrade kernel from OSS and OTN i.e.:

http://www.oracle.com/technetwork/server-storage/linux/downloads/index-088143.html

You can use following commands to install drivers. To use the up2date you should register the system in the Unbreakable Linux Network if you have not registered then download the RPM's from above link and install using rpm -ivh

# up2date -i oracleasm-support oracleasmlib oracleasm-`uname -r`

4) Check to determine that these packages are installed correctly:

# rpm -qa | grep oracleasm

oracleasm-2.6.18-128.1.1.0.1.el5xen-2.0.5-1.el5

oracleasmlib-2.0.4-1.el5

oracleasm-support-2.1.2-1.el5

Now the kernel and the ASMLib should have same version

5) Check the newly installed ASM Driver

# ls -l /lib/modules/$(uname -r)/kernel/drivers/addon/oracleasm

total 120

-rw-r--r-- 1 root root 116168 Jan 10 2015 oracleasm.ko

V. Reboot server to test and verify the new kernel (System Administrator will do this.):

Note: To ensure complete and proper system initialization, revert any temporary file modifications previously implemented, such as /boot/grub/grub.conf, /etc/inittab, /etc/fstab. Where necessary, reboot the system again to verify proper and expected on-boot load of all intended drivers.

Proceed to verify successful network connectivity, storage accessibility, etc. Once verified, further verify successful application functionality, usage and performance.

1) The package versions should match. Run the following command after booting to verify that the oracleasm kernel module matches and will load correctly:

# rpm -q oracleasm-`uname -r`

oracleasm-2.6.18-128.1.1.0.1.el5xen-2.0.5-1.el5

VI. Setup oracleasm (as root):

1) Configure oracleasm

# /etc/init.d/oracleasm configure

Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library

driver. The following questions will determine whether the driver is

loaded on boot and what permissions it will have. The current values

will be shown in brackets ('[]'). Hitting without typing an

answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: grid ---> use GI/ASM user name

Default group to own the driver interface []: asmadmin ---> use GI/ASM group name

Start Oracle ASM library driver on boot (y/n) [n]: y

Fix permissions of Oracle ASM disks on boot (y/n) [y]: y

Writing Oracle ASM library driver configuration [ OK ]

Creating /dev/oracleasm mount point [ OK ]

Loading module "oracleasm" [ OK ]

Mounting ASMlib driver filesystem [ OK ]

Scanning system for ASM disks [ OK ]

2) Verify oracleasm is installed (run on each node, if RAC):

# /etc/init.d/oracleasm status

Checking if ASM is loaded: yes

Checking if /dev/oracleasm is mounted: yes

OR

# /etc/init.d/oracleasm status

Checking if ASM is loaded: running

Checking if /dev/oracleasm is mounted: running

OR

# service oracleasm status

Checking if ASM is loaded: yes

Checking if /dev/oracleasm is mounted: yes

3) Scan ASM disks:

# /etc/init.d/oracleasm scandisks

Scanning system for ASM disks [ OK ]

4) List ASM disks:

# /etc/init.d/oracleasm listdisks

GRIDREP_0001

# ls -l /dev/oracleasm/disks/

total 0

brw-rw---- 1 grid asmadmin 105, 1 Jan 22 16:18 GRIDREP_0001

5) Check the ASM device IDs:

for i in `/etc/init.d/oracleasm listdisks`

do

v_asmdisk=`/etc/init.d/oracleasm querydisk -d $i | awk '{print $2}'`

v_minor=`/etc/init.d/oracleasm querydisk -d $i | awk -F[ '{print $2}'| awk -F] '{print $1}' | awk -F, '{print $1}'`

v_major=`/etc/init.d/oracleasm querydisk -d $i | awk -F[ '{print $2}'| awk -F] '{print $1}' | awk -F, '{print $2}'`

v_device=`ls -la /dev | grep $v_minor | grep $v_major | awk '{print $10}'`

echo "ASM disk $v_asmdisk based on /dev/$v_device [$v_minor $v_major]"

done

ASM disk "GRIDREP_0001" based on /dev/ [105 1]

6) Check the file, /etc/sysconfig/oracleasm:

# cat /etc/sysconfig/oracleasm

Ensure that this file has the same values as the /etc/sysconfig/oracleasm-_dev_oracleasm.orig that was copied in step "2) Make a backup copy of the file, /etc/sysconfig/oracleasm:".

Check all values...for multipathing, especially values for ORACLEASM_SCANORDER and ORACLEASM_SCANEXCLUDE.

If the values are not the same (due to upgrade), then, modify/save the now over-written file, /etc/sysconfig/oracleasm-_dev_oracleasm to be the same as the /etc/sysconfig/oracleasm-_dev_oracleasm.orig.

VII. At this point, Oracle recommends running the "$ORACLE_HOME/bin/relink all" command for the Oracle binaries:

NOTE: Ideally, updated libraries also would work fine with Oracle binaries as OS vendors maintain backward compatibility...but it wouldn't hurt to relink...it also checks missing RPMs.

For CRS home, there is no need to relink the Clusterware binaries.

VIII. Enable all previously disabled services (as root):

1) Enable HA for auto startup:

# /opt/grid/product/11.2.0.3/bin/crsctl enable has

2) Check /etc/inittab and uncomment out respawn of ohasd:

1. # vi /etc/inittab

2. Uncomment out (remove #) the following line in /etc/inittab:

h1:35:respawn:/etc/init.d/init.ohasd run >/dev/null 2>&1 </dev/null

and then run "init q" as the the root user

X. Reboot server to test and verify ASM disks:

a) # rpm -qa | grep oracleasm

oracleasm-2.6.18-128.1.1.0.1.el5xen-2.0.5-1.el5

oracleasmlib-2.0.4-1.el5

oracleasm-support-2.1.2-1.el5

b) # /etc/init.d/oracleasm status

Checking if ASM is loaded: yes

Checking if /dev/oracleasm is mounted: yes

OR

# /etc/init.d/oracleasm status

Checking if ASM is loaded: running

Checking if /dev/oracleasm is mounted: running

OR

# service oracleasm status

Checking if ASM is loaded: yes

Checking if /dev/oracleasm is mounted: yes

c) # /etc/init.d/oracleasm listdisks

GRIDREP_0001

d)

for i in `/etc/init.d/oracleasm listdisks`

do

v_asmdisk=`/etc/init.d/oracleasm querydisk -d $i | awk '{print $2}'`

v_minor=`/etc/init.d/oracleasm querydisk -d $i | awk -F[ '{print $2}'| awk -F] '{print $1}' | awk -F, '{print $1}'`

v_major=`/etc/init.d/oracleasm querydisk -d $i | awk -F[ '{print $2}'| awk -F] '{print $1}' | awk -F, '{print $2}'`

v_device=`ls -la /dev | grep $v_minor | grep $v_major | awk '{print $10}'`

echo "ASM disk $v_asmdisk based on /dev/$v_device [$v_minor $v_major]"

done

ASM disk "GRIDREP_0001" based on /dev/ [105 1]

XI. Verify all ASM disks mounted (as ASM user):

SELECT NAME, DISK_NUMBER, MODE_STATUS, STATE, MOUNT_STATUS, HEADER_STATUS,PATH from v$asm_disk;