ASM Start Failing with ORA-29701 and ORA-01078

[grid@testdb1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Wed Apr 12 22:39:01 2017

Copyright (c) 1982, 2011, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup;

ORA-01078: failure in processing system parameters

ORA-29701: unable to connect to Cluster Synchronization Service

[grid@testdb1 ~]$ asmcmd

Connected to an idle instance.

ASMCMD> ls

ASMCMD-8102: no connection to ASM; command requires ASM to run

Reason:

CSS didn't start because ASM resource was stopped before reboot.

Since ASM was stopped before the node reboot, it will not start afterwards because TARGET=OFFLINE and AUTO_START=restore. Therefore CSS won't start either.

[grid@testdb1 ~]$ crsctl stat res -p

NAME=ora.cssd

AUTO_START=never

-----------------------

NAME=ora.FIRST.dg

AUTO_START=never

-----------------------

NAME=ora.LISTENER.lsnr

AUTO_START=restore

-----------------------

NAME=ora.TEST.dg

AUTO_START=never

-----------------------

NAME=ora.asm

AUTO_START=restore

[grid@testdb1 ~]$ crsctl stat res -t

--------------------------------------------------------------------------------

NAME TARGET STATE SERVER STATE_DETAILS

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.LISTENER.lsnr ONLINE ONLINE testdb1

ora.asm OFFLINE OFFLINE testdb1 Instance Shutdown

ora.ons OFFLINE OFFLINE testdb1

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.cssd 1 ONLINE OFFLINE

ora.diskmon 1 OFFLINE OFFLINE

ora.evmd 1 ONLINE ONLINE testdb1

Solution:

Start ASM using 'srvctl start asm' or by starting any resource (diskgroup, DB) that depends on ASM

[grid@testdb1 ~]$ srvctl start asm;

[grid@testdb1 ~]$ crsctl stat res -t

--------------------------------------------------------------------------------

NAME TARGET STATE SERVER STATE_DETAILS

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.LISTENER.lsnr ONLINE ONLINE testdb1

ora.asm ONLINE ONLINE testdb1 Started

ora.ons OFFLINE OFFLINE testdb1

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.cssd 1 ONLINE ONLINE testdb1

ora.diskmon 1 OFFLINE OFFLINE

ora.evmd 1 ONLINE ONLINE testdb1