A strange issue I encountered recently started with what seemed like a routine preparation task, but quickly turned into a “how did we get here?” kind of situation.
An EXACC Image Upgrade to version 25.2.x was planned, and as part of the prerequisites, all databases had to be brought up to the latest 12.1.0.2 patch level (July 2022 Bundle Patch). Pretty standard requirement, nothing unusual on paper.
So, For a 12.1.0.2 database, The plan was straightforward: apply Patch 34057733 (12.1.0.2.220719 Database Bundle Patch) and move on to the Exacc Image Upgrade.
But as every DBA knows, patching is rarely just “apply and done.”
So I started the action plan with the test environment which is not running on EXACC.
Before applying the bundle patch, I had to deal with existing one-off patches. Some of them were conflicting with the bundle patch (34057733) , so they needed to be rolled back. The idea was simple:
Clean up conflicting patches, bring the Oracle Home to a consistent state, and then apply the latest supported patch level(34057733) followed by datapatch.
A textbook approach… at least in theory.
The opatch lspatches output was as follows before patching took place.
[oracle@testdb dbhome_5]$ OPatch/opatch lspatches33568947;33566041;19239846;21322448;19472320;27051384;22747454;21321429;24007737;22731026;22828765;21967332;21864513;32327201;31335037;30432084;32327208;31858428;UPDATE PERL IN 12.1.0.2 DATABASE ORACLE HOME TO V5.3233192628;Database PSU 12.1.0.2.211019,Oracle JavaVM Component (OCT2021)33197468;JDK BUNDLE PATCH 12.1.0.2.21101921923026;31413047;33114885;Database Bundle Patch : 12.1.0.2.211019 (33114885)32758932;OCW PATCH SET UPDATE 12.1.0.2.210720 (32758932)OPatch succeeded.
The patch had conflicts with several existing one-off patches:
- 19239846
- 21923026
- 24007737
- 33566041
- 33568947
- 33066573
- 33318861
Nothing unusual so far. This is part of the job.
[oracle@testdb setup] $ORACLE_HOME/OPatch/opatch rollback -id 19239846[oracle@testdb setup] $ORACLE_HOME/OPatch/opatch rollback -id 21923026[oracle@testdb setup] $ORACLE_HOME/OPatch/opatch rollback -id 24007737[oracle@testdb setup] $ORACLE_HOME/OPatch/opatch rollback -id 33566041[oracle@testdb setup] $ORACLE_HOME/OPatch/opatch rollback -id 33568947[oracle@testdb setup] $ORACLE_HOME/OPatch/opatch rollback -id 33066573[oracle@testdb setup] $ORACLE_HOME/OPatch/opatch rollback -id 33318861[oracle@testdb 34057733]$ $ORACLE_HOME/OPatch/opatch apply
As expected, I started rolling them back one by one. The process was smooth and predictable. Each rollback completed successfully, and everything looked under control.
I also applied Patch 34057733 (12.1.0.2.220719 Database Bundle Patch). Again, there were no errors during the patching phase, so I proceeded with datapatch. However, datapatch failed.
In the logs, I encountered the following error messages:
ORA-00439: feature not enabled: Database queuingORA-06512: at "SYS.DBMS_AQADM_SYS", line 13568
As soon as I saw these errors, I checked whether the Advanced Queuing option was enabled by querying V$OPTION. The result was shocking, all database options were shown as FALSE, which is clearly not normal.
It looked like a relink failure. Although opatch completed successfully, I ran relink all, but all the options in V$OPTION were still FALSE.
Since this was a test environment, I re-ran the same patching steps on the Oracle Home and checked V$OPTION after each step to determine at which point the options were being set to FALSE.
After rolling back patch 33318861, the issue was observed.
At first, there was no dramatic failure. No obvious crash. But the database started behaving in a way that didn’t make sense.
The symptoms was:
All entries in V$OPTION → FALSE
Database → not functioning properly
datapatch → cannot be executed
And yet…
DBA_REGISTRY → ALL VALID
Let that sink in.
From the registry perspective, everything looked perfectly fine, no invalid components, no invalid objects, no warning signs.
But in reality, the database was effectively broken.
It was one of those deeply unsettling situations where Oracle tells you, “everything is valid”, while at the same time proving the exact opposite.
Out of curiosity, I re-applied patch 33318861.
Immediately:
V$OPTION → back to TRUE
Database → fully functional again
DBA_REGISTRY Represents:
- SQL-level component state
- Catalog metadata
V$OPTION Represents:
- Binary-level capabilities
- Features enabled during linking of the Oracle executable
What actually happened was that rolling back patch 33318861 likely triggered a relink of the Oracle binaries. During this process, the required option flags and linked components were lost, resulting in Oracle incorrectly reporting that no options were enabled. This explains why V$OPTION showed all values as FALSE and why certain database features became unavailable, leading to instability. Interestingly, DBA_REGISTRY remained unchanged, so from that perspective everything still appeared to be VALID, even though the underlying functionality was effectively broken.
Patch 33318861: MERGE REQUEST ON TOP OF DATABASE BP 12.1.0.2.211019 FOR BUGS 32968733 (Patch)
Bugs Resolved by this Patch was
- 21117072 DBREPLAY PATCH BUNDLE 1 FOR 12.1.0.2
- 22611167 PDB DEFAULT SERVICE SHOULD NOT STORE FQ NAME IN SERVICE$ AND CDB_SERVICE$
- 23182225 DPLOAD.SQL EXECUTES AN SQLPLUS RECONNECT, LEADING TO PROBLEMS WITH DBMS_SQLPATCH
- 25139545 TRACKING BUG TO INCLUDE DPLOAD.SQL FROM MAIN FOR FIXES ON 1120X AND 1210X
- 26196152 TRACKING BUG FOR CONSOLIDATED CLOUD INFRA FIXES FOR DB (DBCS,EXACS,BMIAAS)
- 27487279 ADM EXPORT CMD FAILING – INCORRECT BACKPORT FOR BUG 22826718 IN JAN 2018 BP/PSU
- 28538439 PTIME FOR SYS NOT GETTING UPDATED AFTER APPLYING JULY PSU
- 29667994 OKV18.1: WHEN SECRET HSM_PASSWORD IS ADDED WHILE USING LOCAL WALLET, AUTOLOGIN DOESN’T WORK
According to the bugs resolved by this patch, fix 26196152 appears to be related, as it is a tracking bug for consolidated cloud infrastructure fixes for database environments (DBCS, EXACS, and BM IaaS). Since the database home was originally cloned from an Exadata Cloud@Customer (EXACC) environment, rolling back patch 33318861 likely introduced a deeper issue by effectively corrupting the Oracle Home. After that point, applying patch 34057733 did not recover the database home, as the equivalent fix from the original baseline seems to be missing in this newer bundle.
At this stage, it became clear that this was not a simple patching inconsistency, but rather a gap between patch levels and missing fix integration across versions. To investigate further, I opened an SR and explained the full scenario to the support engineer, including the rollback behavior, the relink impact, and the unexpected loss of enabled database options. I specifically requested guidance on whether the same fix could be re-applied on top of 34057733, since the original correction was delivered on top of 12.1.0.2.211019, and that context no longer aligned with the current patch level.
With the help of the support engineer and after involving the development team, it was advised by Dev that patch 34345244 should be applied on top of 12.1.0.2.220719. Patch 34345244 also contains the same merged fixes and is expected to address the missing or inconsistent components introduced during the rollback scenario.
Also according to the readme document Patch 34345244: MERGE REQUEST ON TOP OF DATABASE BP 12.1.0.2.220719 FOR BUGS 34076214, Bugs Resolved by this Patch was :
- 21117072 DBREPLAY PATCH BUNDLE 1 FOR 12.1.0.2
- 22611167 PDB DEFAULT SERVICE SHOULD NOT STORE FQ NAME IN SERVICE$ AND CDB_SERVICE$
- 23182225 DPLOAD.SQL EXECUTES AN SQLPLUS RECONNECT, LEADING TO PROBLEMS WITH DBMS_SQLPATCH
- 25139545 TRACKING BUG TO INCLUDE DPLOAD.SQL FROM MAIN FOR FIXES ON 1120X AND 1210X
- 26196152 TRACKING BUG FOR CONSOLIDATED CLOUD INFRA FIXES FOR DB (DBCS,EXACS,BMIAAS)
- 27487279 ADM EXPORT CMD FAILING – INCORRECT BACKPORT FOR BUG 22826718 IN JAN 2018 BP/PSU
- 28538439 PTIME FOR SYS NOT GETTING UPDATED AFTER APPLYING JULY PSU
- 29667994 OKV18.1: WHEN SECRET HSM_PASSWORD IS ADDED WHILE USING LOCAL WALLET, AUTOLOGIN DOESN’T WORK
After applying the patch 34076214, all features in v$option are turned to TRUE again and the issue has been solved but datapatch failed with the different following error:
[oracle@testdb 34345244]$ $ORACLE_HOME/OPatch/datapatch -verboseSQL Patching tool version 12.1.0.2.0 Production on Thu Feb 19 13:51:02 2026Copyright (c) 2012, 2017, Oracle. All rights reserved.Log file for this invocation: /orahome/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_1754113_2026_02_19_13_51_02/sqlpatch_invocation.logConnecting to database...OKBootstrapping registry and package to current versions...done Error in bootstrap log /orahome/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_1754113_2026_02_19_13_51_02/bootstrap1_AKTLIVE.log: Error at line 89: Warning: Package Body created with compilation errors. Error at line 95: 49/13 PLS-00323: subprogram or cursor 'SESSION_INITIALIZE' is declared Error in bootstrap log /orahome/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_1754113_2026_02_19_13_51_02/bootstrap1_AKTLIVE.log: Error at line 89: Warning: Package Body created with compilation errors. Error at line 95: 49/13 PLS-00323: subprogram or cursor 'SESSION_INITIALIZE' is declaredPrereq check failed, exiting without installing any patches.Please refer to MOS Note 1609718.1 and/or the invocation log/orahome/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_1754113_2026_02_19_13_51_02/sqlpatch_invocation.logfor information on how to resolve the above errors.SQL Patching tool complete on Thu Feb 19 13:51:04 2026
SYS.DBMS_SQLPATCH was in invalid state. The error was “Body 49 13 PLS-00323: subprogram or cursor ‘SESSION_INITIALIZE’ is declared in a package specification and must be defined in the package body”
I ran the prvtsqlpatch.plb script below, as the DBMS_SQLPATCH creation script was included within it.
SQL> @?/rdbms/admin/prvtsqlpatch.plbSession altered.PL/SQL procedure successfully completed.Table created.Warning: Package Body created with compilation errors.Errors for PACKAGE BODY DBMS_SQLPATCH:LINE/COL ERROR-------- -----------------------------------------------------------------49/13 PLS-00323: subprogram or cursor 'SESSION_INITIALIZE' is declared in a package specification and must be defined in the package bodySession altered.SQL> alter package SYS.DBMS_SQLPATCH compile body;Warning: Package Body altered with compilation errors.SQL> show errorsErrors for PACKAGE BODY SYS.DBMS_SQLPATCH:LINE/COL ERROR-------- -----------------------------------------------------------------49/13 PLS-00323: subprogram or cursor 'SESSION_INITIALIZE' is declared in a package specification and must be defined in the package bodySQL> select OWNER, OBJECT_NAME, OBJECT_TYPE, STATUS from dba_objects where OBJECT_NAME='DBMS_SQLPATCH';OWNER--------------------------------------------------------------------------------OBJECT_NAME--------------------------------------------------------------------------------OBJECT_TYPE STATUS----------------------- -------SYSDBMS_SQLPATCHPACKAGE VALIDSYSDBMS_SQLPATCHPACKAGE BODY INVALIDOWNER--------------------------------------------------------------------------------OBJECT_NAME--------------------------------------------------------------------------------OBJECT_TYPE STATUS----------------------- -------PUBLICDBMS_SQLPATCHSYNONYM VALID
But this did not resolve the issue. A new problem then emerged: the SESSION_INITIALIZE function was defined with five input parameters, however in its wrapped body there was no corresponding implementation handling a function with five input parameters.
PROCEDURE session_initialize(p_oh IN VARCHAR2, p_force IN BOOLEAN := FALSE, p_debug IN BOOLEAN := FALSE, p_attempt IN NUMBER := NULL, p_use_rollback_files IN BOOLEAN := FALSE);
The body and the specification of the SESSION_INITIALIZE function were not consistent. To address this, I replaced the existing files by copying the latest versions of dbmssqlpatch.sql and prvtsqlpatch.plb from the extracted 34057733 bundle patch setup into the $ORACLE_HOME/rdbms/admin directory.
[oracle@testdb Setup]$ find /orahome/app/Setup -type f -name "prvtsqlpatch.plb" -exec ls -ls {} \;12 -rw-r--r-- 1 oracle oinstall 10428 Jul 22 2022 /orahome/app/Setup/34057733/31965033/files/rdbms/admin/prvtsqlpatch.plb12 -rw-r--r-- 1 oracle oinstall 9363 Jul 22 2022 /orahome/app/Setup/34057733/29141038/files/rdbms/admin/prvtsqlpatch.plb12 -rw-r--r-- 1 oracle oinstall 9029 Jul 22 2022 /orahome/app/Setup/34057733/24340679/files/rdbms/admin/prvtsqlpatch.plb8 -rw-r--r-- 1 oracle oinstall 4553 Jul 22 2022 /orahome/app/Setup/34057733/20243804/files/rdbms/admin/prvtsqlpatch.plb12 -rw-r--r-- 1 oracle oinstall 9339 Jul 22 2022 /orahome/app/Setup/34057733/24732088/files/rdbms/admin/prvtsqlpatch.plb8 -rw-r--r-- 1 oracle oinstall 7165 Jul 22 2022 /orahome/app/Setup/34057733/23144544/files/rdbms/admin/prvtsqlpatch.plb12 -rw-r--r-- 1 oracle oinstall 9359 Jul 22 2022 /orahome/app/Setup/34057733/29972716/files/rdbms/admin/prvtsqlpatch.plb8 -rw-r--r-- 1 oracle oinstall 5226 Jul 22 2022 /orahome/app/Setup/34057733/20594149/files/rdbms/admin/prvtsqlpatch.plb12 -rw-r--r-- 1 oracle oinstall 9853 Jul 22 2022 /orahome/app/Setup/34057733/30364137/files/rdbms/admin/prvtsqlpatch.plb8 -rw-r--r-- 1 oracle oinstall 5262 Jul 22 2022 /orahome/app/Setup/34057733/21527488/files/rdbms/admin/prvtsqlpatch.plb12 -rw-r--r-- 1 oracle oinstall 10213 Jul 22 2022 /orahome/app/Setup/34057733/31511219/files/rdbms/admin/prvtsqlpatch.plb12 -rw-r--r-- 1 oracle oinstall 10708 Jul 22 2022 /orahome/app/Setup/34057733/34057733/files/rdbms/admin/prvtsqlpatch.plb[oracle@testdb Setup]$ find /orahome/app/Setup -type f -name "dbmssqlpatch.sql" -exec ls -ls {} \;4 -rw-r--r-- 1 oracle oinstall 3777 Jul 22 2022 /orahome/app/Setup/34057733/20243804/files/rdbms/admin/dbmssqlpatch.sql8 -rw-r--r-- 1 oracle oinstall 5970 Jul 22 2022 /orahome/app/Setup/34057733/24732088/files/rdbms/admin/dbmssqlpatch.sql8 -rw-r--r-- 1 oracle oinstall 5407 Jul 22 2022 /orahome/app/Setup/34057733/23144544/files/rdbms/admin/dbmssqlpatch.sql8 -rw-r--r-- 1 oracle oinstall 4596 Jul 22 2022 /orahome/app/Setup/34057733/20594149/files/rdbms/admin/dbmssqlpatch.sql8 -rw-r--r-- 1 oracle oinstall 6113 Jul 22 2022 /orahome/app/Setup/34057733/31511219/files/rdbms/admin/dbmssqlpatch.sql
I simply copied the latest versions of both files, but first I backed up the existing versions in the $ORACLE_HOME/rdbms/admin directory.
[oracle@testdb admin]$ mv dbmssqlpatch.sql dbmssqlpatch_backup.sql[oracle@testdb admin]$ mv prvtsqlpatch.plb prvtsqlpatch_backup.plb[oracle@testdb admin]$ cp dbmssqlpatch.sql /orahome/app/Setup/34057733/31511219/files/rdbms/admin/dbmssqlpatch.sql[oracle@testdb admin]$ cp prvtsqlpatch.plb /orahome/app/Setup/34057733/34057733/files/rdbms/admin/prvtsqlpatch.plb
These two versions of the files were fully consistent with each other. It was a complex issue that required deep investigation. After that, datapatch ran successfully, and all operations were finally completed without errors.
Hope it helps.


Leave your comment