Category Archives: Oracle Database Appliance

My Oracle Database Server PS1 Environment Variable

Today I was doing some tricky remote troubleshooting of an ORDS deployment. It turned out that it was the networking team doing fun stuff with the removal of cookies from only one of the multiple load-balanced devices between the users and the ORDS server (99% sure of this, we’ll see later if I was right). Since this stuff can be almost impossible to figure out without sniffing packets and/or visibility into the whole network configuration, a good trick to remember is to just deploy ORDS standalone on the desktop of the user attempting to troubleshoot things. If it works on their desktop, then it’s almost certainly the networking team doing something funny between the users and the application servers.

During the whole troubleshooting session, where the DBA was typing in all the commands as I dictated them (so much fun!), the DBA was constantly typing pwd. This post is for that DBA.

I’ve added the following line to my .bash_profile on all my Oracle database servers:

export PS1='\n[\u@\h DB:$( echo $ORACLE_SID ) \t \w]\n\$ '

PS1 is the environment variable that controls what your prompt looks like.

Mine has the following:

\n = a new line. This means that all of my commands have some nice whitespace (although I use green on black, so is that blackspace?) between them.

[ = the starting bracket for my prompt.

\u@\h = username@hostname

DB:$( echo $ORACLE_SID ) = Each time a new prompt is generated, find the value of the $ORACLE_HOME environment variable and display it with a DB: in front of it. It took me a bit to figure out how to get this to evaluate every time a new prompt was drawn. For my middleware servers, I leave this portion off. I suppose if your middleware server talks to only a single database, you could just hardcode it into the prompt.

\t = the current time

\w = the current working directory. No more typing pwd every other command!

] = the ending bracket for my prompt.

\n = another new line. This allows your actual command prompt to always start on the left side of the screen instead of constantly moving across the screen as you navigate down directory paths.

\$ = print the normal prompt. Notice that there is a space before the closing ‘ so that commands are not jammed up against the prompt.

Hopefully, an Oracle DBA or two out there finds this useful!

Happy DBAing!

#joelkallmanday


Oracle 12.1.2.7 Virtualized ODA Install

I’ve been working with the brand new 12.1.2.7 ODA update on an X5-2 and Oracle has made a number of changes that impact how an install and configuration is done.

12.1.2.7 is the first release that supports a ‘reflash’ of an ODA with Oracle Linux 6. Previously update 12.1.2.5 had Oracle Linux 5 and updated 12.1.2.6 had Oracle Linux 6, but the only way to move to 12.1.2.6 was to first already be at 12.1.2.5 and then patch to 12.1.2.6.

12.1.2.7 is the first release that will support (re-)deployment of an ODA in a virtualized configuration where you load the Oracle VM ISO into each ILOM and then deploy the ODA_BASE virtual machine. Note that if your ODA was at 12.1.2.6 or earlier you will still have to download the 12.1.2.7 patch to ODA_BASE after you deploy ODA_BASE to patch some of the server and storage components. Oracle has changed from the old patch –infra, –grid, –database options and now has a –server (that does some hardware and the grid infrastructure) and –storage (which patches the shared storage hardware).

One very major change which is especially important for those of us who manage virtualized ODA installs remotely is that /etc/xen/xend-config.sxp on ODA_BASE now has the following section:

# The interface for VNC servers to listen on. Defaults
# to 127.0.0.1  To restore old 'listen everywhere' behaviour
# set this to 0.0.0.0
#(vnc-listen '127.0.0.1')

This means that you when you attempt to create a VM and connect to the console the same way you did in past ODA versions you’re probably going to get something like this:

[root@oda-base-node0 ~]# oakcli show vmconsole OL7U2 
 OAKERR:8006 Error in VNC display configurations : DISPLAY=localhost:10.0

You can either modify the /etc/xen/xend-config.sxp file, or you can update the vm.cfg file for a particular VM that you’ve imported. Since the ODA will be using ACFS it might be a bit hard to find the vm.cfg file since it ends up in the hidden .ACFS directory on the shared repository ACFS file system.

I had created my VMs on a shared repository named sharedrepo so my vm.cfg file ended up here:

[root@oda-base-node0 OL7U2]# pwd 
 /u01/app/sharedrepo/sharedrepo/.ACFS/snaps/OL7U2/VirtualMachines/OL7U2 
[root@oda-base-node0 OL7U2]# vim vm.cfg

Update the vm.cfg file and add the following:

vnclisten = '0.0.0.0'

Note that vnclisten ‘0.0.0.0’ didn’t work and neither did vnclisten = 0.0.0.0. It had to be vnclisten = ‘0.0.0.0’ for the VM to start.

Now that you’ve got a ‘listen anywhere’ vnc console running, you can just use a VNC client on your workstation (I used Tiger VNC) and connect to your node0-dom0:590X where X is the number of VMs you’ve started past ODA_BASE.  ODA_BASE is at :5900 and if this is the first additional VM you’ve created you’ll be able to reach the console at :5901. Note that once you’ve done the initial network configuration of your VM you’ll probably want to take the vnclisten parameter out of the vm.cfg file.

I’m also currently working with Oracle support on an issue. We’ve got a RAC One Node multitenant database and catcon.pl won’t run scripts in the ‘root container and all pluggables’. For example:

[oracle@oda-base-node0 ~]$ $ORACLE_HOME/perl/bin/perl $ORACLE_HOME/rdbms/admin/catcon.pl -u SYS -d $ORACLE_HOME/rdbms/admin -b utlrp_output utlrp.sql 
 catcon: ALL catcon-related output will be written to utlrp_output_catcon_77673.lst 
 catcon: See utlrp_output*.log files for output generated by scripts 
 catcon: See utlrp_output_*.lst files for spool files, if any 
 Enter Password: 
 catconInit: database is not open on the default instance 
 Unexpected error encountered in catconInit; exiting

This is important because apxremov_con.sql is just a wrapper for catcon.pl so you can’t remove APEX from the root container (CDB$ROOT) and PDB$SEED so that new pluggable databases won’t have APEX in them. We do this so that we’ll be able to potentially have different versions of APEX in different pluggable databases in the same container database.

Oracle has opened a P1 bug on this and has actively been working this for a few days.

Happy ODAing!

Rich


Is your ODA disk shelf randomly disconnecting from your compute nodes?

I’ve been doing a lot of work with ODAs (Oracle Database Appliance) lately and I have to say that I’m VERY impressed. It’s amazing how smooth Oracle can make things when they know exactly what hardware and software will be on a machine. I can’t imagine how long it would take to patch an ILOM, BIOS, SSDs, HDs, OS, various controllers, Grid Infrastructure, and Database Software, etc. if you were doing everything by scratch. With an ODA all of that can be done by downloading two files and running a few commands.

We did run into an ‘interesting’ situation where the disk shelf would randomly disconnect from the compute nodes. A restart of ohasd would bring everything back for a while (weeks, to sometimes hours), but it was really troubling to say the least. After trying a whole bunch of things, Oracle finally asked us to take a picture of the back of our disk shelves… See those service ports below? Someone, at some point in time, had plugged Ethernet cables into those ports. And that was the issue..

Those ports are only used during initial machine configuration and should NOT be used on a running machine. What would happen is a buffer would fill up and/or get some kind of packet that it didn’t know what to do with and the controller would reset. Of course the same thing would happen to the other controller at the same time. If you lose both controllers to your disk shelf simultaneously, bad things tend to happen to your database…

As far as we can tell those cables were plugged in during installation and it took over a year before the resets happened!

The below is from an X4-2, your ODA might be a bit different.

X4-2ODA