It Finally Happened For Me!

I’ve been working on an upgrade of Oracle Database on Windows. Despite working with Oracle Database for over 30 years, I really never spent a whole lot of time working on a Windows server. Unix, Solaris, Linux, heck even AIX, oh yeah. Windows, not so much.

While attempting to patch a brand new software-only install of Oracle 19c from the original 19.3 up to 19.21 I kept on getting UtilSession failed: Prerequisite check “CheckActiveFilesAndExecutables” failed during my opatch apply. It appeared that the JDK home in my ORACLE_HOME was in use. Of course, this didn’t make any sense since there wasn’t anything running out of this home.

Here’s what I was seeing:

Following active files/executables/libs are used by ORACLE_HOME: c:\app\oracle\product\19.0.0.0\dbhome_1

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\bin\java.exe

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\java.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\management.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\msvcr100.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\net.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\nio.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\server\jvm.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\verify.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\zip.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\lib\ext\cldrdata.jar

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\lib\ext\localedata.jar

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\lib\ext\zipfs.jar

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\lib\jsse.jar

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\lib\rt.jar

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\bin\java.exe

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\java.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\management.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\net.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\nio.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\verify.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\zip.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\bin\server\jvm.dll

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\lib\jsse.jar

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\lib\rt.jar

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\lib\ext\cldrdata.jar

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\lib\ext\localedata.jar

c:\app\oracle\product\19.0.0.0\dbhome_1\jdk\jre\lib\ext\zipfs.jar

Why were they in use? Why were they listed twice? What Windows process had a lock on them? I couldn’t figure it out.

I reached out to the system admins for the Windows box to learn that the machine had anti-virus software. Pete Magee has a very well-written blog post about running anti-virus software on your Oracle Database server. Of course that was it! I got them to turn it off, but it didn’t fix the issue. Same exact error, same exact message.

Time for drastic measures. I downloaded IObit’s Unlocker program (use at your own risk) and used it to look at the files. It was showing no locks on any of those files, but I went ahead and unlocked them anyway. Of course this was it. This had to fix it… Nope! Still broken.

In desperation, I reached out to Oracle Support. And that’s when it finally happened for me. Support generated this:

My Oracle Support has performed a Knowledge search using your Service Request details (Summary, Error codes, Product) and located the following documents which may be related to your issue.



Search Results

====================================================================================

99% - Doc ID 2507120.1 Opatch 11.2.0.3.20 : Applying PSU/Windows BP fails with:'To run in silent mode, OPatch requires a response file for Oracle Configuration Manager (OCM)', 'error code = 73'

99% - Doc ID 1475147.1 OPatch - Failed to load the patch object. Possible causes are: OPatch failed with error code = 73 LsInventorySession failed:

99% - Doc ID 2950970.1 opatch apply fails with Error:" Prerequisite check "CheckActiveFilesAndExecutables" failed" on $oracle_home/jdk files

99% - Doc ID 1472242.1 Applying Patch Fails as Copy Failed for crsctl.bin as EM dbconsole was not Stopped

99% - Doc ID 2978449.1 "opatch util verify" reports OUI-67073:UtilSession failed: Files are not updated completely & OUI-67124:Files check failed: Some files under oracle_home are not patched , after applying RU Patches

Wait… That third Doc ID looks exactly like my error. I took a look and started reading and while it didn’t match my situation, it did mention the problem. Someone had downloaded opatch for the wrong platform. Could I really have done that? You betcha! (been watching a lot of Fargo with my wife, and “Minnesotan” is very catchy).

I got a new version of OPatch, this time for Windows instead of for Linux and you know what? Everything worked just fine.

In over 30 years of using Oracle Support, this is the first time I’ve ever had the “Oracle Support did a search and we found these documents” actually have the solution to my problem!

Happy New Year and Happy DBAing!