How does DB Agent Discover Nodes in an Oracle RAC Architecture

[#1]

How does DB Agent Discover Nodes in an Oracle RAC Architecture

This guide explains how the DB agent discovers nodes within an Oracle RAC architecture. An understanding of this will help with troubleshooting when the DB agent runs into issues during this process.

Prerequisites

• System requirements: 512 MB of min and max heap per node/instance + 1GB

• License requirements: 1 License unit per node. Note: Disabling a collector consumes 1 License unit irrespective of the number of nodes

• Enable auto discovery of the nodes through the JVM flag shown below. This will allow the DB agent to discover the nodes through the process shown in this article. If the value is true, the agent will monitor the DB as a standalone using the collector configuration.

-Ddbagent.oracle.cluster.discovery.disabled="false"

Note: DB agent will determine the flow by extracting details using all queries shown below from the DB server. This logic is inbuilt in the agent.

DB Agent Flow Before Node Discovery Occurs

1. An initial connection is made by using the connection details provided in the collector
2. If the initial connection is successful, DB agent will verify if DB is cluster-based using the query shown below
   1. If value is true –> start node discovery
   2. If value is false –> monitor DB using the initial connection (from step 1)

SELECT value FROM v$parameter WHERE name='cluster_database’.

 

DB Agent Flow While Node Discovery Occurs for Cluster-Based DB

1. Verify if cluster is a PDB or CDB type using the below query (the values in this query will be used during the mode connections that are mentioned later in this article)

select decode(sys_context('USERENV', 'CON_NAME'),'CDB$ROOT',sys_context('USERENV', 'DB_NAME'), sys_context('USERENV', 'CON_NAME')) DB_NAME, decode(sys_context('USERENV','CON_ID'),0,'CDB',1,'CDB','PDB') TYPE, sys_context('USERENV', 'SERVICE_NAME') service_name from DUAL​

*Oracle Document for reference: https://docs.oracle.com/en/database/oracle/oracle-database/19/multi/viewing-information-about-cdbs-a...

2. Retrieve nodes from the cluster:

select i.inst_id, i.instance_name, utl_inaddr.get_host_address(i.host_name), p.value from gv$instance i LEFT JOIN  gv$parameter p ON (p.inst_id = i.inst_id AND p.name = 'local_listener');

If above query is unsuccessful, use the one below:

select i.inst_id, i.instance_name, i.host_name, p.value from gv$instance i LEFT JOIN  gv$parameter p ON (p.inst_id = i.inst_id AND p.name = 'local_listener');

* The values from local listener is derived from listener.ora file, located in $ORACLE_HOME directory.

3. Using the output from above, DB agent will try to connect to each individual node using the different modes described below

• If type is PDB, service name from the query in Step 1 will be used
• If type is CDB, instance name from the query in Step 2 will be used
• DB agent will make a connection using the host, port, and instance or service name. See below table for more details on where the host and port is coming from
• Each mode connection will be tried sequentially until there is a successful one

Connection Type	Host name from Collector Config	Host Name from Query (value from column 3)	Host Name from Query (value from TNS entry - column 4)	Port from Collector Config	Port from Query (value from TNS entry - column 4 or default port used if empty)
Mode1		✔️		✔️
Mode2			✔️		✔️
Mode3		✔️			✔️
Mode4	✔️			✔️

* Pass  JVM argument -Ddbagent.oracle.cluster.discovery.mode={Mode1 or Mode2, etc} during agent startup to only use a specific mode connection. Not recommended unless a specific connection is required

4.  If none of the modes are successful, monitoring will not occur

Troubleshooting 

Most issues seen in the agent logs are related to connection issues at a collector level or at a node level. Check the connection for each node using the username of the DB agent. Refer to the table to understand which host and port to use.

telnet [host] [port]

curl -v [host]:[port]

Conclusion

By reading this guide, you should have a better understanding of how DB agent flows when an Oracle RAC collector type has been created. This should help with troubleshooting if issues arise with the help of your DBA. If you continue to experience issues, please contact support for further assistance.