Friday, October 6, 2017

ORA-29701: unable to connect to Cluster Synchronization Service during ASM startup

ORACLE  Version : 12.2 standalone with ASM

Trying to start ASM instance after server reboot and it is failing with error
    
oradev.domain.com:+ASM:/opt/oracle/12.1.0/grid/bin $ sqlplus / as sysasm

SQL*Plus: Release 12.1.0.2.0 Production on Fri Oct 6 16:19:59 2017

Copyright (c) 1982, 2014, Oracle.  All rights reserved.

Connected to an idle instance.


SQL> startup
ORA-01078: failure in processing system parameters
ORA-29701: unable to connect to Cluster Synchronization Service
SQL> exit
Disconnected

oradev.domain.com:+ASM:/opt/oracle/12.1.0/grid/bin $ ./crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.asm
               OFFLINE OFFLINE      oradev.domain.com                   Instance Shutdown,STABLE
ora.ons
               OFFLINE OFFLINE      oradev.domain.com                   STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        OFFLINE OFFLINE                               STABLE
ora.diskmon
      1        OFFLINE OFFLINE                               STABLE
ora.evmd
      1        ONLINE  ONLINE       oradev.domain.com                   STABLE

                 


oradev.domain.com:+ASM:/opt/oracle/12.1.0/grid/bin $ ./crsctl start resource ora.cssd
CRS-4995:  The command 'Start  resource' is invalid in crsctl. Use srvctl for this command

oradev.domain.com:NOTSET:/opt/oracle/12.1.0/grid/bin $ ps -ef | grep cssd.bin
oracle   12249     1  0 16:21 ?        00:00:26 /opt/oracle/12.1.0/grid/bin/ocssd.bin
oracle   21138 20798  0 21:49 pts/0    00:00:00 grep cssd.bin

oradev.domain.com:+ASM:/opt/oracle/12.1.0/grid/bin $ ./crsctl start resource ora.cssd -unsupported
CRS-2672: Attempting to start 'ora.cssd' on 'oradev.domain.com'
CRS-2672: Attempting to start 'ora.diskmon' on 'oradev.domain.com'
CRS-2676: Start of 'ora.diskmon' on 'oradev.domain.com' succeeded
CRS-2676: Start of 'ora.cssd' on 'oradev.domain.com' succeeded


oradev.domain.com:+ASM:/opt/oracle/12.1.0/grid/bin $ ./crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.asm
               OFFLINE OFFLINE      oradev.domain.com                   Instance Shutdown,STABLE
ora.ons
               OFFLINE OFFLINE      oradev.domain.com                   STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  ONLINE       oradev.domain.com                   STABLE
ora.diskmon
      1        OFFLINE OFFLINE                               STABLE
ora.evmd
      1        ONLINE  ONLINE       oradev.domain.com                   STABLE

                 
                 
oradev.domain.com:+ASM:/opt/oracle/12.1.0/grid/bin $ sqlplus / as sysasm

SQL*Plus: Release 12.1.0.2.0 Production on Fri Oct 6 16:23:58 2017

Copyright (c) 1982, 2014, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ASM instance started

Total System Global Area 2147483648 bytes
Fixed Size                  2926472 bytes
Variable Size            2111002744 bytes
ASM Cache                  33554432 bytes
ASM diskgroups mounted


ASM successfully started …………………

Here "AUTO_START" is set to "NEVER" and that is the reason for not starting after reboot but we can change to "ALWAYS" to avoid this Manual intervention after every reboot

$ ./crsctl stat res ora.cssd -p -init | grep AUTO_START
AUTO_START=never

Refer:
ASM Instance doesn't start in Oracle Restart (Standalone GI) environment after node reboot (Doc ID 1917176.1)

Auto Scroll Stop Scroll