Recover after a failed switchover
Problem Description
You might find yourself in a situation where your switchover fails and left your environment with two physical standby databases.
From the broker your might see the following error:
ORA-16816: incorrect database role
Solution
1. Logon (as sysdba) to the instance that was your Primary database instance before the switchover.
2. Confirm the database role.
SQL> select database_role from v$database;
DATABASE_ROLE
-------------------------------
PHYSICAL STANDBY
3. Shutdown the instance.
SQL> shutdown immediate;
Oracle Instance Shutdown.
4. Mount the database.
SQL> startup mount;
Database opened.
5. Cancel the MRP process. You will receive “ORA-16136: Managed Standby Recovery not active” if it is not running, but you can ignore.
SQL> alter database recover managed standby database cancel;
6. Terminate the current switchover to Standby that never completed fully.
SQL> alter database recover managed standby database finish;
Database altered
7. Now switchover to Primary.
SQL> alter database commit to switchover to primary with session shutdown;
Database altered
8. Open the database.
SQL> alter database open;
Database altered.
9. Confirm the database role.
SQL> select database_role from v$database;
DATABASE_ROLE
------------------------------
PRIMARY
Additional Steps
When attempting to open the Primary Database you may suffer the following error:
SQL> alter database open
*
ERROR at line 1:
ORA-16649: possible failover to another database prevents this database being opened
In this case, before you can open the database, you must disable Data Guard Broker as follows:
SQL> alter system set dg_broker_start=false scope=both;
System altered.
SQL> alter database open;
Database altered.
Now re-install Data Guard Broker.