EM 12c Emctl Start Oms Fails with Error Oracle Management Server is Down

Problem:

[oracle@stagedbms3 oracle]$ $OMS_HOME/bin/emctl start oms

Oracle Enterprise Manager Cloud Control 12c Release 12.1.0.1.0

Starting WebTier...

WebTier Successfully Started

Starting Oracle Management Server...

Oracle Management Server Successfully Started

Oracle Management Server is Down

The Following error messages are observed in emctl.log and EMGC_OMS1.out respectively.

=> emctl.log

2016-05-30 13:52:49,387 [main] DEBUG oms.StatusOMSCmd pingOMSCheck.341 - Attempting to connect to https://stagedbms3.ipx.com:7302/em/faces/logon/core-uifwk-console-login

2016-05-30 13:52:50,287 [main] DEBUG oms.StatusOMSCmd pingOMSCheck.341 - Attempting to connect to https://stagedbms3.ipx.com:7803/empbs/check

2016-05-30 13:52:50,308 [main] DEBUG oms.StatusOMSCmd processStatusOMS.240 - console page status code is 404

2016-05-30 13:52:51,428 [main] DEBUG oms.StatusOMSCmd processStatusOMS.242 - pbs page status code is 500

2016-05-30 13:52:51,428 [main] DEBUG oms.StatusOMSCmd processStatusOMS.245 - console header is null

2016-05-30 13:52:51,428 [main] DEBUG oms.StatusOMSCmd processStatusOMS.247 - pbs header is null

2016-05-30 13:52:51,428 [main] DEBUG oms.StatusOMSCmd processStatusOMS.285 - Both Console and PBS are down. Status codes are: Console - 404 and PBS - 500

2016-05-30 13:52:51,428 [main] INFO oms.StatusOMSCmd processStatusOMS.294 - Failed to connect to em login page

2016-05-30 13:52:51,429 [main] INFO oms.StatusOMSCmd processStatusOMS.295 - Encountered error while hitting page. Status codes are 404 for console and 500 for PBS

2016-05-30 13:52:51,429 [main] INFO oms.StatusOMSCmd processStatusOMS.296 - Connection exception when trying to hit OMS page. OMS is down

=> EMGC_OMS1.out

INFO: Found persistence provider "org.eclipse.persistence.jpa.PersistenceProvider". OpenJPA will not be used.

[EL Info]: 2016-05-30 14:14:30.244--ServerSession(1290639949)--EclipseLink, version: Eclipse Persistence Services - 2.1.3.v20110304-r9073

[EL Severe]: 2016-05-30 14:14:31.255--ServerSession(1290639949)--Local Exception Stack:

Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.1.3.v20110304-r9073): org.eclipse.persistence.exceptions.DatabaseException

Internal Exception: weblogic.jdbc.extensions.ConnectionDeadSQLException: weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error:

The Network Adapter could not establish the connection

Caused by: weblogic.jdbc.extensions.ConnectionDeadSQLException: weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error: The Network Adapter could not establish the connection

Solution:

OMS failing to establish the connection the repository database. TO start up OMS successfully, both repository database and listener should be up and running.

How to fix it:

1. Verify that the Repository Database and Listener are up and running.

2. Check if the Repository database is registered with the listener using that command:

$ lsnrctl services

3. Verify that you can connect to Repository Database as 'SYSMAN' using SQLPLUS.

If the problem still persists, follow the steps below:

4. Shutdown the OMS

$<OMS_HOME>/bin/emctl stop oms -force

5. Stop the Repository Database and its listener.

6. Start in this order:

the Listener, and the Repository Database.

7. Start the OMS :

$<OMS_HOME>/bin/emctl start oms

Oracle Support Doc ID 1394880.1