Unable to Parse the Content of exa_map Error on ExaCC Grid RU Precheck

Error on ExaCC Grid Release Update Precheck

While trying to update grid release on an Exadata Cloud@Customer system by using GUI, which is not I am not mucly accustomed to, it failed on first precheck phase and warned us checking ssh connectivity between nodes.

As a solution I followed the “How to Manage and Check Health Exadata Cloud Related Services – ExaCS & ExaCC (Doc Id 2926648.1)“

Step 1: First check passwordless ssh – working for the opc / grid / oracle OS Users

Run the command provided below for the opc / grid / oracle OS Users.

[opc@exaccnode1 ~]$ for node in `sudo /u01/app/19.0.0.0/grid/bin/olsnodes`; do
  ssh -t -t \
      -i /home/opc/.ssh/id_rsa \
      -o StrictHostKeyChecking=no \
      opc@"$node" \
      'hostname -s; date'
done

Step 2: Display Details About the Exadata Cloud System

Check platform information:

[root@exaccnode1 ~]# /var/opt/oracle/misc/platforminfo
  EXACS
[root@exaccnode2 ~]# /var/opt/oracle/misc/platforminfo
  EXACS

Step 3: Check rpm packages

[root@exaccnode1 ~]# rpm -qa --last | egrep -i 'dbaastools_exa|dbcs'
  dbcs-agent-update-exacc-25.1.2.0.0-250505.1725.x86_64 Tue 20 May 2025 08:59:17 PM +03
  dbcs-admin-update-exacc-25.1.1.0.0-250220.1206.x86_64 Thu 20 Mar 2025 07:45:25 PM +03
  dbastools_exa-1.0-23.3.1.0.0-230817.2029.x86_64 Sun 08 Oct 2023 06:15:21 PM +03
  dbcs-agent-exacc-2.8OL7-22.3.1.1.0-221013.0027.x86_64 Mon 27 Mar 2023 05:07:26 PM +03
[root@exaccnode2 ~]# rpm -qa --last | egrep -i 'dbaastools_exa|dbcs'
  dbcs-agent-update-exacc-25.1.2.0.0-250505.1725.x86_64 Tue 21 May 2025 10:26:40 PM +03
  dbcs-admin-update-exacc-25.1.1.0.0-250220.1206.x86_64 Thu 20 Mar 2025 08:54:50 PM +03
  dbastools_exa-1.0-23.3.1.0.0-230817.2029.x86_64 Sun 08 Oct 2023 06:15:30 PM +03
  dbcs-agent-exacc-2.8OL7-22.3.1.1.0-221013.0027.x86_64 Mon 27 Mar 2023 05:07:40 PM +03

Step 4: Check Database Cloud Service Agent Status

[root@exaccnode1 ~]# systemctl status dbcsagent
...
Main PID : 88785 (code exited, status=127)
systemd[1]: Unit dbcsagent.service entered failed state.
[root@exaccnode2~]# systemctl status dbcsagent

Step 5: Checking the Database Cloud Service Agent Logs

[root@exaccnode1 ~]# cat /opt/oracle/dcs/log/dcsagent-stderr.log
  /bin/bash: /u01/oracle.dbcsagent/graalvm-jre-22.0.2/bin/java: No such file or directory

Java binary was not available under the graalvm-jre directory.

I have copied it from the healthy node.

[opc@exaccnode2 ~]$ scp java exaccnode1:/u01/oracle.dbcsagent/graalvm-jre-22.0.2/bin/

After retrying, the error message changed, but the service still did not start.

[root@exaccnode1 ~]# cat /opt/oracle/dcs/log/dcsagent-stderr.log
  Error: could not open `/u01/oracle.dbcsagent/graalvm-jre-22.0.2/lib/jvm.cfg`

The file counts were not consistent across the nodes. One node had 28 files, while the other had 43.

[root@exaccnode1 ~]# cd /u01/oracle.dbcsagent/graalvm-jre-22.0.2/lib/
[root@exaccnode1 lib]# ls -als | wc
  28     272   1657
[root@exaccnode2 ~]# cd /u01/oracle.dbcsagent/graalvm-jre-22.0.2/lib/
[root@exaccnode2 lib]# ls -als | wc
  28     422   2593
  
[opc@exaccnode2 graalvm]$ scp -r lib exaccnode1:/u01/oracle.dbcsagent/graalvm-jre-22.0.2/
[root@exaccnode1 ~]# systemctl restart dbcsagent

I copied the lib directory from the healthy node. After that, I was able to restart the dbcs agent service.

After starting the dbcs agent service, I retried the grid release update precheck and failed again but this time with a different error :

[FATAL][DBAAS-70382]Unable to Parse the Content of exa_map from URL

This was the error we encountered, and at the time there was no clear explanation or documented solution—neither from Oracle Support nor from publicly available sources.

Initially, I attempted to locate the exa_map file. I discovered that exa_map resides in the location specified by the oss_container_url parameter in the exadbcpatch.cfg file, located under /var/opt/oracle/exapatch.

The oss_container_url defines an Oracle Object Storage (OSS) container used by Exadata to download and upload patch-related artifacts and metadata.

[root@exaccnode1 ~]# cat /var/opt/oracle/exapatch/exadbcpatch.cfg | grep oss_container_url
oss_container_url="http://169.254.100.27:2080/dbaas_patch/shome"

I tried to download the file using curl and inspect its contents to understand why it could not be parsed, but this did not provide any useful insight.

[root@exaccnode1 ~]# curl -v -o http://169.254.100.27:2080/dbaas_patch/shome/exa_map

Step 6: Install the latest version of dbaastools

After several unsuccessful attempts and unnecessary troubleshooting,I finally ran the command shown below which should have been the very first step. Simply updating the dbaascli tools resolved the issue. This is a step that must be performed before starting any patching operation.

[root@exaccnode1 ~]# dbaascli admin updateStack

Hope it helps.