Thursday, May 16, 2013

Oracle 11gR2 DBFS and Peopletools 8.53 (bis)

Few days ago I wrote an article about Oracle 11gR2 DBFS and Peopletools 8.53.
As I said out there, there are 3 mains problems that I see :
* first, the database may grow dramatically
* second, the WebServer and FileSystem host must be Linux
* third, the fuse does not allow the automount even though you configure it in /etc/fstab. And automount would not be a solution anyway since on server boot the mount is happening before the autostart of the database which needs to be up to mount such a DBFS…

Despite those disadvantages, it is worth to go there, we get a more secure way to manage the reports.

We cannot do much on the second point, or wait for Oracle 12c (?) to unlock the OS limitation and offer this solution to all the shops regardless their OS.
On the first point, we can manage it, it’s all about how you deal with your space devices (db files, archives and backup).
And on the third point, it is just a matter of workaround…

The all difficulty in a Peoplesoft environment is that the filesystem used for the reporting must be reachable as soon as AppServer/Batch server and PIA start, otherwise some remaining batches may not post their reports.
So, whether we have to mount the DBFS after the start of the database, we also have to mount it before the start of the Peoplesoft processes. If we configured all your environment to start automatically, we cannot do that manually, it would be done once the system is available, too late for the AppServer and other Peoplesoft processes.

Here we’ll go through a solution how to implement a mount of the DBFS without manual intervention.

Note that all the tests below are done on Peoplesoft Appliance from April-2013, HCM92 Peopletools 8.53.02.
1. Go through the all configuration to be done as I explained in my other blog entry, Oracle 11gR2 DBFS and Peopletools 8.53.  This configuration is the bare minimum to make the manual mount working, but still required here. Following steps are in addition to make the “auto-mount”.

2. Set your libraries environment, and load:

[root@hcm92000 ~]# echo "/usr/local/lib" >> /etc/ld.so.conf.d/usr_local_lib.conf
[root@hcm92000 ~]# ln -s /u01/app/oracle/product/11.2.0.x/db_1/lib/libclntsh.so.11.1 /usr/local/lib/libclntsh.so.11.1 [root@hcm92000 ~]# ln -s /u01/app/oracle/product/11.2.0.x/db_1/lib/libnnz11.so /usr/local/lib/libnnz11.so
[root@hcm92000 ~]# ln -s /lib64/libfuse.so.2 /usr/local/lib/libfuse.so.2
[root@hcm92000 ~]# ldconfig


3. Add a line corresponding to your mount point in /etc/fstab

[root@hcm92000 ~]# echo "/u01/scripts/dbfsmount#sysadm@HR92DM00 /mnt/dbfs fuse rw,user,noauto,direct_io 0 0" >> /etc/fstab
[root@hcm92000 ~]# more /etc/fstab
LABEL=/                 /                       ext3    defaults        1 1
LABEL=/boot             /boot                   ext3    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults,size=2g        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
LABEL=SWAP-VM           swap                    swap    defaults        0 0
LABEL=Oracle11gR2       /u01                    ext3    defaults        1 2
LABEL=HCMDB             /opt/oracle/psft/ptdb   ext3    defaults        1 2
LABEL=TOOLS             /opt/oracle/psft/pt     ext3    defaults        1 2
LABEL=SES             /opt/oracle/psft/ses     ext3    defaults        1 2
/u01/scripts/dbfsmount#sysadm@HR92DM00 /mnt/dbfs fuse rw,user,noauto,direct_io 0 0
[root@hcm92000 ~]#


4. Create the script to mount the DBFS
In the /etc/fstab defined on step 3, I specified /u01/scripts/dbfsmount which is a script as following (SYSADM being the accessid’s password):
[root@hcm92000 ~]# cd /u01/scripts/
[root@hcm92000 scripts]# more dbfsmount
#!/bin/bash
export ORACLE_HOME=/u01/app/oracle/product/11.2.0.x/db_1
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/usr/lib:$LD_LIBRARY_PATH;export LD_LIBRARY_PATH
nohup $ORACLE_HOME/bin/dbfs_client $@ -o allow_other << PSWD &
SYSADM
PSWD

!! Do not forget the option “-o allow_other”, otherwise psadm2 won’t be granted for DBFS access (the file /etc/fuse.conf must contains a line with the value: user_allow_other). This script is called when the mount on /mnt/dbfs is raised on the system.
Then give the proper rights to be executed on “mount” command:
chmod 750 /u01/scripts/dbfsmount
chgrp fuse /u01/scripts/dbfsmount


5. Modify the dbstart script
In the directory /u01/scripts, you have the dbstart script which needs to be modified to run the mount command. Modify it as following, here is just an extract (in bold the added lines, actually just after the db startup):
[root@hcm92000 scripts]# grep mount -B 10 -A 10 dbstart
    fi
    su -s /bin/bash  $ORACLE_OWNER -c "$ORACLE_HOME/bin/sqlplus -s /nolog" <<SQL
connect / as sysdba
alter system set LOCAL_LISTENER='(ADDRESS = (PROTOCOL=TCP)(HOST=$(hostname))(PORT=$DEFAULT_LISTENER_PORT))';
exit;
SQL
    su -s /bin/bash  $ORACLE_OWNER <<MOUNT
if [ $ORACLE_SID = "HR92DM00" ]; then
echo "Mounting DBFS... "$ORACLE_SID
cd ~
mount /mnt/dbfs
fi
MOUNT
}

stop_db_inst () {
    ret=0
    export ORACLE_SID ORACLE_HOME
    su -s /bin/bash  $ORACLE_OWNER -c "$ORACLE_HOME/bin/sqlplus -s /nolog" <<SQL
connect / as sysdba
shutdown immediate
[root@hcm92000 scripts]#

I made a test on the database name, just to make thing clear that I want the dbfs mounted for that peculiar database. However, it is not required on Peoplesoft Appliance since there’s only one database.

6. Reboot the server
Now it’s ready. Time to reboot the server and see.
You may see a warning regarding the /etc/fstab format. It's a blessing in disguise, I’d say.
DBFS_009
And during the boot, you’ll see the echo message about the DBFS mounting (the nohup line can be ignored):
DBFS_010

7. Check the DBFS filesystem availability:
[psadm2@hcm92000 ~]$ ls -lrt /mnt/dbfs/ReportRepository/HR92DM00
total 0
drwxr-xr-x 6 psadm2 oracle 0 May 14 11:42 20130514
[psadm2@hcm92000 HR92DM00]$

Here we can see only one day of reports.

8. Run a test, for instance AEMINITEST and check once again the DBFS filesystem:

[psadm2@hcm92000 ~]$ ls -lrt /mnt/dbfs/ReportRepository/HR92DM00
total 0
drwxr-xr-x 6 psadm2 oracle 0 May 14 11:42 20130514
drwxr-xr-x 3 psadm2 oracle 0 May 16 04:29 20130516
[psadm2@hcm92000 HR92DM00]$

The current date has been added. The posting was successful.

9. Double check the all configuration works from the db side:

[oracle@hcm92000 ~]$  export ORACLE_SID=HR92DM00
[oracle@hcm92000 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Thu May 16 04:30:17 2013
Copyright (c) 1982, 2011, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning and Oracle Label Security options

SQL> set lines 200 pages 100
SQL> select pathname from sysadm.T_REPORTREPOSITORY where filedata is not null;

PATHNAME
------------------------------------------------------
/HR92DM00/20130514/861/AE_AEMINITEST_1126.stdout
/HR92DM00/20130514/863/sysaud01_1128.out
/HR92DM00/20130514/863/SQR_SYSAUD01_1128.log
/HR92DM00/20130514/863/sysaud01_1128.PDF
/HR92DM00/20130514/862/SQR_DDDAUDIT_1127.log
/HR92DM00/20130514/862/dddaudit_1127.PDF
/HR92DM00/20130514/862/dddaudit_1127.out
/HR92DM00/20130514/866/AE_AEMINITEST_1131.stdout
/HR92DM00/20130516/867/AE_AEMINITEST_1132.stdout <—here is my last posted report


As of now, we don’t have to worry about the DBFS mount anymore.
So, why not have this configuration by default on future Peoplesoft Appliances and/or Peoplesoft OVM if any. It’s rather a good test to have on a demo.

Enjoy,

Nicolas.

Tuesday, May 14, 2013

Oracle 11gR2 DataBase FileSystem (DBFS) and Peopletools 8.53

Coming with Oracle 11gR2, the DBFS (DataBase FileSystem) feature improves the files storage.
So far, we had BLOB, files stored in the database, but we needed an access to the database with specific API to read/open them.
With DBFS, files are still stored in the database, but access may be from ‘outside’ the database, like for any other filesystem.

Peoplesoft can take all the advantages of this feature, especially useful for the batch reports. As of Peopletools 8.53, it’s now possible.
Storing the batch reports in such way, files being in the database (but still readable from outside the database), can make the administrator life easier regarding the backup. No need to worry anymore about backing up the report repository directory, it’s already done from within the database backup !
The process scheduler (batch server) will continue to post in the defined report repository (now the DBFS), and the database will handle it automatically.

Rather nice enhancement.

Important note, all the tests below are done on the Peoplesoft Appliance from April-2013, HCM9.2 Peopletools 8.53.02. Your situation may differ, so adapt your scripts and environment accordingly.

In the current situation, when posting, files are going to a pre-define directory in the configuration.properties, such as :
[psadm2@hcm92000 ~]$ ls -lrt /home/psadm2/psft/pt/8.53/psreports/HR92DM00/
total 24
drwxr-xr-x 7 psadm2 oracle 4096 Apr 24 07:38 20130424
drwxr-xr-x 5 psadm2 oracle 4096 Apr 27 08:49 20130427
drwxr-xr-x 3 psadm2 oracle 4096 May  8 04:49 20130508
drwxr-xr-x 3 psadm2 oracle 4096 May 13 11:13 20130513
You have to manage a backup all those files to be ready for a restore, if any.

Let’s have a look how to configure and use the DBFS now. Again, the main goal here is to avoid external file to manage backup for.

1. The kernel
If not installed, download and install the rpm kernel-devel-2.6.18-274.el5.i686 (http://public-yum.oracle.com):
[root@hcm92000 fuse-2.7.4]# more /etc/redhat-release
Red Hat Enterprise Linux Server release 5.7 (Tikanga)
[root@hcm92000 fuse-2.7.4]#
[root@hcm92000 Oracle_11.2.0.3.0_64bits]# rpm -q kernel-devel-2.6.18-274.el5.i686
kernel-devel-2.6.18-274.el5
[root@hcm92000 Oracle_11.2.0.3.0_64bits]#

2. Fuse
2.1 Download fuse utility, http://sourceforge.net/projects/fuse/files/fuse-2.X/2.7.4/
2.2 Install fuse :
[root@hcm92000 Oracle_11.2.0.3.0_64bits]# ls
fuse-2.7.4.tar.gz   database  
[root@hcm92000 Oracle_11.2.0.3.0_64bits]# gunzip fuse-2.7.4.tar.gz
[root@hcm92000 Oracle_11.2.0.3.0_64bits]# tar xvf fuse-2.7.4.tar
fuse-2.7.4/
...<snipped>
[root@hcm92000 fuse-2.7.4]# ls /usr/src/kernels
2.6.18-274.el5-i686
[root@hcm92000 fuse-2.7.4]# ./configure --prefix=/usr --with-kernel=/usr/src/kernels/2.6.18-274.el5-i686
checking build system type... x86_64-unknown-linux-gnu
...<snipped>
NOTE:     Detected that FUSE is already present in the kernel, so
NOTE:     building of kernel module is disabled.  To force building
NOTE:     of kernel module use the '--enable-kernel-module' option.
configure: creating ./config.status
config.status: creating Makefile
config.status: creating config.h
[root@hcm92000 fuse-2.7.4]#
[root@hcm92000 fuse-2.7.4]# make
...<snipped>
[root@hcm92000 fuse-2.7.4]# make install
...<snipped>
[root@hcm92000 fuse-2.7.4]#
[root@hcm92000 fuse-2.7.4]# /sbin/depmod
[root@hcm92000 fuse-2.7.4]# /sbin/modprobe fuse
[root@hcm92000 fuse-2.7.4]# chmod 666 /dev/fuse
[root@hcm92000 fuse-2.7.4]# echo "/sbin/modprobe fuse" >> /etc/rc.modules
[root@hcm92000 fuse-2.7.4]#
  chmod 700 /etc/rc.modules

3. The tablespace
Connect on the Peoplesoft database, create the tablespace:
[oracle@hcm92000 ~]$ export ORACLE_SID=HR92DM00
[oracle@hcm92000 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Tue May 14 03:54:08 2013

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning and Oracle Label Security options

SQL> show parameter db_securefile

NAME                                 TYPE                                         VALUE
------------------------------------ -------------------------------------------- ------------db_securefile                        string                                       PERMITTED
SQL>
SQL> create tablespace dbfs_tbs datafile '/u01/app/oracle/oradata/HR92DM00/dbfs_tbs_01.dbf' size 500M reuse autoextend on next 200M segment space management auto;

Tablespace created.

SQL> grant dbfs_role to sysadm;

4. The filesystem
Connect on the database with your accessid (SYSADM) and create the filesystem (dbfs_tbs is the tablespace, ReportRepository will be the filesystem):
[oracle@hcm92000 ~]$ ls $ORACLE_HOME/rdbms/admin/dbfs*
/u01/app/oracle/product/11.2.0.x/db_1/rdbms/admin/dbfs_create_filesystem_advanced.sql  /u01/app/oracle/product/11.2.0.x/db_1/rdbms/admin/dbfs_drop_filesystem.sql
/u01/app/oracle/product/11.2.0.x/db_1/rdbms/admin/dbfs_create_filesystem.sql
[oracle@hcm92000 ~]$ sqlplus sysadm

SQL*Plus: Release 11.2.0.3.0 Production on Tue May 14 03:57:56 2013

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Enter password:

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning and Oracle Label Security options

SQL> @$ORACLE_HOME/rdbms/admin/dbfs_create_filesystem.sql dbfs_tbs ReportRepository
No errors.
--------
CREATE STORE:
begin dbms_dbfs_sfs.createFilesystem(store_name => 'FS_REPORTREPOSITORY',
tbl_name => 'T_REPORTREPOSITORY', tbl_tbs => 'dbfs_tbs', lob_tbs => 'dbfs_tbs',
do_partition => false, partition_key => 1, do_compress => false, compression =>
'', do_dedup => false, do_encrypt => false); end;
--------
REGISTER STORE:
begin dbms_dbfs_content.registerStore(store_name=> 'FS_REPORTREPOSITORY',
provider_name => 'sample1', provider_package => 'dbms_dbfs_sfs'); end;
--------
MOUNT STORE:
begin dbms_dbfs_content.mountStore(store_name=>'FS_REPORTREPOSITORY',
store_mount=>'ReportRepository'); end;
--------
CHMOD STORE:
declare m integer; begin m := dbms_fuse.fs_chmod('/ReportRepository', 16895);
end;
No errors.
SQL>

5. The mount point
5.1 As root, create the path directory first to be mounted within the create filesystem:
[root@hcm92000 fuse-2.7.4]# mkdir -p /mnt/dbfs
[root@hcm92000 fuse-2.7.4]# chmod -R a+rwx /mnt/dbfs

5.2 As root, create a new fuse group
[root@hcm92000 ~]# export PATH=/usr/sbin:$PATH;export PATH
[root@hcm92000 ~]# groupadd fuse

5.3 As root, add the fuse group to oracle and psadm2 (application/batch/pia server domain owner)
[root@hcm92000 ~]# usermod -G dba,fuse oracle
[root@hcm92000 ~]# usermod -G fuse psadm2

5.4 As oracle, create a file, /etc/fuse.conf to allow the users to read the new moint point:
[oracle@hcm92000 ~]$ more /etc/fuse.conf
user_allow_other

5.5 Put the accessid’s password in a file (SYSADM by default)
[oracle@hcm92000 ~]$ echo "SYSADM" > dbfspassword

5.6 As oracle user on your OS, mount the filesystem:
[oracle@hcm92000 ~]$ LD_LIBRARY_PATH=$ORACLE_HOME/lib:/usr/lib;export LD_LIBRARY_PATH
[oracle@hcm92000 ~]$ nohup dbfs_client SYSADM@HR92DM00 /mnt/dbfs -o allow_other < dbfspassword &
[1] 13293
[oracle@hcm92000 ~]$ nohup: appending output to `nohup.out'
[oracle@hcm92000 ~]$ more nohup.out
Password:
[oracle@hcm92000 ~]$

5.7 Check your moint points, you should see the new one:
[oracle@hcm92000 ~]$ mount
/dev/sda2 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/sda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw,size=2g)
/dev/sdb1 on /u01 type ext3 (rw)
/dev/sdc1 on /opt/oracle/psft/ptdb type ext3 (rw)
/dev/sdd1 on /opt/oracle/psft/pt type ext3 (rw)
/dev/sde1 on /opt/oracle/psft/ses type ext3 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
192.168.1.1:/software on /nfs/software type nfs (rw,addr=192.168.1.1)
dbfs-SYSADM@HR92DM00:/ on /mnt/dbfs type fuse (rw,nosuid,nodev,max_read=1048576,default_permissions,allow_other,user=oracle)

And the directory:
[oracle@hcm92000 ~]$ ls -lrt /mnt/dbfs/ReportRepository/
total 0
[oracle@hcm92000 ~]$


6. Change the Report repository of webserver
Connect as psadm2, pia owner, modify the variable ReportRepositoryPath in configuration.properties.
Shutdown your PIA, modify the file, and restart the PIA, such as:
[psadm2@hcm92000 bin]$ ./stopPIA.sh
Submitting shutdown command for WebLogic Server PIA at t3://hcm92000.phoenix.nga:8000

No activity will be logged to this window.
Server activity will be logged to /home/psadm2/psft/pt/8.53/webserv/peoplesoft/servers/PIA/logs/PIA_shutdown*

Stopping Weblogic Server...
Done
[psadm2@hcm92000 bin]$ vi ../applications/peoplesoft/PORTAL.war/WEB-INF/psftdocs/ps/configuration.properties
# Report Repository file path
# This is the installation default entered on the screen in setup.exe
# If one is not specified in the Web Profile then this is used.
## qc="7", sd="Report Repository File Path", rq="N", tip="This location is only used if this detail is not specified in the Web Profile", ld="This is the path to store PeopleSoft Reports"
#ReportRepositoryPath=/home/psadm2/psft/pt/8.53/psreports
ReportRepositoryPath=/mnt/dbfs/ReportRepository
[psadm2@hcm92000 bin]$ ./startPIA.sh
Attempting to start WebLogic Server PIA
No activity will be logged to this window.
Server activity will be logged to /home/psadm2/psft/pt/8.53/webserv/peoplesoft/servers/PIA/logs/PIA_*
PID for WebLogic Server PIA is: 11782
[psadm2@hcm92000 bin]$ tail -f /home/psadm2/psft/pt/8.53/webserv/peoplesoft/servers/PIA/logs/PIA_stdout.log
...
<May 14, 2013 4:11:32 AM EDT> <Notice> <WebLogicServer> <BEA-000329> <Started WebLogic Admin Server "PIA" for domain "peoplesoft" running in Production Mode>
<May 14, 2013 4:11:32 AM EDT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to RUNNING>
<May 14, 2013 4:11:32 AM EDT> <Notice> <WebLogicServer> <BEA-000360> <Server started in RUNNING mode>
...

7. Now ready to test
From within the front end, run processes, for instances AEMINITEST, DDDAUDIT and/or SYSAUDIT. Standard procedure here, I won’t go through the screenshots for this step. I assume it is well known by the readers. You should be able to read the reports like anytime before. The change should be pretty transparent for the end users.

8. Check from the back end
From the OS side, you may check whether the files are there or not (it’s not the original location as shown earlier):
[psadm2@hcm92000 LOGS]$  ls -lrt /mnt/dbfs/ReportRepository/HR92DM00/20130514/*
/mnt/dbfs/ReportRepository/HR92DM00/20130514/861:
total 1
-rw-r--r-- 1 psadm2 oracle 292 May 14 07:52 AE_AEMINITEST_1126.stdout

/mnt/dbfs/ReportRepository/HR92DM00/20130514/863:
total 7
-rw-r--r-- 1 psadm2 oracle    0 May 14 08:11 sysaud01_1128.out
-rw-r--r-- 1 psadm2 oracle 1757 May 14 08:11 SQR_SYSAUD01_1128.log
-rw-r--r-- 1 psadm2 oracle 4309 May 14 08:11 sysaud01_1128.PDF

/mnt/dbfs/ReportRepository/HR92DM00/20130514/862:
total 7
-rw-r--r-- 1 psadm2 oracle 1757 May 14 08:12 SQR_DDDAUDIT_1127.log
-rw-r--r-- 1 psadm2 oracle 4401 May 14 08:12 dddaudit_1127.PDF
-rw-r--r-- 1 psadm2 oracle    0 May 14 08:12 dddaudit_1127.out
[psadm2@hcm92000 LOGS]$

And from the database side, you can query the dedicated table to your new filesystem:
SQL> select pathname from sysadm.T_REPORTREPOSITORY
SQL> /

PATHNAME
--------------------------------------------------------------------------------
/
/.sfs
/.sfs/RECYCLE
/.sfs/attributes
/.sfs/content
/.sfs/snapshots
/.sfs/tools
/HR92DM00
/HR92DM00/20130514
/HR92DM00/20130514/861
/HR92DM00/20130514/861/AE_AEMINITEST_1126.stdout
/HR92DM00/20130514/862
/HR92DM00/20130514/862/SQR_DDDAUDIT_1127.log
/HR92DM00/20130514/862/dddaudit_1127.PDF
/HR92DM00/20130514/862/dddaudit_1127.out
/HR92DM00/20130514/863
/HR92DM00/20130514/863/SQR_SYSAUD01_1128.log
/HR92DM00/20130514/863/sysaud01_1128.PDF
/HR92DM00/20130514/863/sysaud01_1128.out

19 rows selected.
You may see, there’s 7 files in there, corresponding to the produced files by the batches (FILEDATA is BLOB datatype):
SQL> select pathname from sysadm.T_REPORTREPOSITORY where filedata is not null;

PATHNAME
--------------------------------------------------------------------------------
/HR92DM00/20130514/861/AE_AEMINITEST_1126.stdout
/HR92DM00/20130514/863/sysaud01_1128.out
/HR92DM00/20130514/863/SQR_SYSAUD01_1128.log
/HR92DM00/20130514/863/sysaud01_1128.PDF
/HR92DM00/20130514/862/SQR_DDDAUDIT_1127.log
/HR92DM00/20130514/862/dddaudit_1127.PDF
/HR92DM00/20130514/862/dddaudit_1127.out

7 rows selected.

9. Potential errors
9.1 You may see when mounting (file nohup):
fuse: bad mount point `/mnt/dbfs': No such file or directory
Verify the directory exists and Oracle has right enough.

9.2 You may receive an error on posting, like :
Java Exception: Error while write to file:java.lang.SecurityException: Directory Creation Failed  (63,49)
Means, you probably forget the fuse group and/or add the file /etc/fuse.conf, and/or forget the option allow_other in the mount command.
Also, verify the parameter ReportRepositoryPath in the configuration.properties of the webserver. I was puzzled for quite a while until I see the value was mistakenly appended by a blank space…

Conclusions:
=> As of now, the database backup should be enough to restore data and reports !
Main side effect, depending of the number and size of the reports your application is producing, your database dedicated tablespace may dramatically grow up, take care about your archives, and backup space required.

=> Two of the requirements may dramatically limit the use: first, your WebServer must run on Linux, second, the DBFS must be hosted on Linux…

=> Also, note that the fuse does not support automount mode (fstab), after server boot, you should do it manually, or write your own script using wallet for authentication… Otherwise, the batch will create the reports directly onto the filesystem /mnt/dbfs, not in the database… making discrepancies between filesystems content and database. Furthermore, you will then get some troubles to get it mounted again.

Well, a nice try, very interesting to know it exists. Not sure though if many shops will implement it. Do you ?

Nicolas.

References:
Peopletools Peoplebooks: http://docs.oracle.com/cd/E38689_01/pt853pbr0/eng/pt/tadm/task_ImplementingTheOracleDatabaseFileSystem.html
Oracle online documentation: http://docs.oracle.com/cd/E11882_01/appdev.112/e18294/adlob_fs.htm

Monday, May 13, 2013

Integration Network WorkCenter 8.53 : automated process

As of Peopletools 8.52, a lot has been done regarding the Integration Broker configuration and use through the Integration Network WorkCenter.
Many tasks have been made easier than ever for the configuration and maintenance of all the messaging between systems.
Few months ago, I showed how it can be set and used.

And even more, from the Peopletools 8.53, it was promised in the RVP that the monitoring could be automated through email:
“Automated Network Status and Real-Time Notification of Errors
In PeopleTools 8.52, checking the status of PeopleSoft systems in the Integration Network was a manual process. Administrators could go to a page in the Integration WorkCenter and see the status of all connected systems. New in 8.53, checking the status is an automated process, and the results of the check can be sent directly to an administrator.
Once configured, the Integration Network can monitor itself for errors.”

So, I was expecting a lot within this new feature. Could we work only with this Peopletools monitoring utility rather than a third party to monitor the messaging ? Let’s have a look.

Here the main page:
INW_853_049 
Of course, the configuration must be set, for more details, please read my previous blog entry (it was for Peopletools 8.52, but still valid in Peopletools 8.53).
Go to the “Network Status Setup” (this has been added in Peopletools 8.53):
INW_853_050
Go to the SetUp, fill up an email:
INW_853_021 
Optionally, include diagnosis and information.
You can propagate the setting to your remote node(s):
INW_853_051  
Now the test. I wrongly configured a message USER_PROFILE between my HR and EP systems, then create a new user on HR.
Few minutes later, the magic is done ! An email was dropped to my inbox:
INW_853_031 
Magic, but rather cryptic, isn’t it ?

Firstly, at this very moment, I do not really care that much the XML input.

Secondly, whether the email subject contains the source node (publisher), I have no idea about the target node (subscriber). I do not have more idea about the application which send my out this email. A dedicated email source must be set up front.

Thirdly, the message “Unable to find a routing corresponding to the incoming request message” is way too standard. We could expect much more, at least the same details as such we can find from within the front-end (see below).

Fourthly, there’s no link to the application. It would be much easier to have such link, give the credentials on a login page and being redirected to the Integration Network WorkCenter.

Last and not least, this email is sent every X minutes (X being the interval defined in the setup of automated integration) until you fix it, or until the limit is reached (last number of days). Should it not be enough to receive the error only once for the last X minutes ? Apparently not, by default it is checking all the messaging remaining in error every 5 minutes, and such for the last 30 days !
During my tests, I was like spammed…

Coming from Peoplesoft to automate its own processes, I would expect much more.
This email input is quite disappointing.

That said, keep the given transactionID and go to the transactional tracker in the front-end:
INW_853_032 
Go to the “Search Details”, and paste the previously TransactionID indentified to be in error from the email:
INW_853_033 
Now, we have a little bit more information about the message:
INW_853_034
Going to the “View Error/Info”, we will retrieve the exact same error we got through the email:
INW_853_035
Nothing really helpful on this page. The previous page is, to my point of view, more helpful, you have the external service name, the publishing node and the service operation. You already know there’s an error, so you should be able to fix it. Such information would be nice in the email.

Now, to go further in my testing of this automated process, I’m shutting down the target messaging server (EP, the subscriber node).
INW_853_036
And create a new user on HR, a message should be send to EP, at least a try. Obviously it shouldn't work.
INW_853_037
Going to the monitoring overview, after a while, the message in “Retry” status:INW_853_039
Going to the details, it clearly states an error:

INW_853_040
The error is quite obvious that time:
INW_853_041 
Unfortunately, I’ve never receive any email for that error ! I’m wondering if that’s because the status of the message is “Retry” and not “Error”.
It can remain endlessly in this status if I’m waiting for the email, unless I connect to the system and check it manually.
Again, disappointed.
Ok, I should read more carefully the RVP that I mentioned at the beginning :
”[…]Once configured, the Integration Network can monitor itself for *errors*.[…]”
It’s clear enough, *error*… But do we not have an error here ? And what’s the meaning of the automation if I have to connect to the front end to check for the other “problematic” status (I mean not “done”) ?

Not regarding the automation of monitoring, but still about the Network Status.
Checking the status by default returns green icons (only if everything is well configured):
INW_853_052
Go to the “Network Status Setup”:
INW_853_053
As said earlier, on this page we can configure the automated monitoring, but there’s also diagnostics section. Check all of them out, save, and return back to the Network Status page. Check it again:
INW_853_054
Now there’s “Warning” on every and each node (drag the mouse on over the triangles) ! We did not have any problem though !
Going to the details:
INW_853_055
And to the diagnostics :
INW_853_056 
Well, fine with me, that can be useful to have details and diagnostics, but why, oh why is it a yellow triangle with warning ?!?!

In the end, my conclusion…

Here has been discussed about:
1. The automated email of Network Integration
2. The monitoring of messages (Retry status is really not monitored?)
3. The Network Integration Status, warning…

I would say none of these points are entirely satisfying my expectations:
1. I would give much more information in the Email content: less cryptic content, more information about the error, give a link to the application.
2. The status of messages to be monitored by the automatically processed should be configurable. Such an email to be send when a message is coming to a given status (checkboxes for the status would have been even better). I don’t really see any advantages of the automation if we have to connect and check for the other status like “Retry”, “Timeout” which are also kind of error.
3. Whether I can understand the need of diagnostic, it’s certainly not a warning ! To me, a warning must always be considered as an error. But here there’s no problem, it is just the result of a configuration setting.

That’s a nice try, but I tend to think this is only a first try to be improved in the next Peopletools release !

Enjoy,

Nicolas

Note: all the tests from the above have been made on the delivered Peoplesoft Appliances from April-2013, HCM9.2 and FSCM9.2 on Peopletools 8.53.02.
References:
1. To configure an email address on your sandbox, you can follow the instructions of Hakan Biroglu: http://bloggingaboutoracleapplications.org/gmail-smtp-server-peoplesoft/
2. The online Peoplebooks have been greatly improved within the Peopletools 8.53, I used it for this blog entry:
http://docs.oracle.com/cd/E38689_01/pt853pbr0/eng/pt/tiba/intro_UsingtheIntegrationNetwork-388000.html

Sunday, April 28, 2013

Virtual Appliance for Peoplesoft – few issues

Hereafter few issues I’ve seen so far on the VirtualBox Appliances for Peoplesoft. Some are on FSCM92000 only, some on both (HCM92 and FSCM92), and some only when you have the two running in same time.
From the errors/issues/problems described below have more or less impact on the system, I’d say one on Samba (the VBOX registration) and the non-working auto-start application on FSCM92 are the most annoying.

FSCM92000 is a VirtualBox Appliance for Peoplesoft, it should be exactly the same as HCM92000 that I showed how it can be moved from VirtualBox to ESXi.
A lot are already configured by default, it can easily be up and running.
However, for some reason, FSCM92000 is a little bit less configured (buggy?).

1) SELinux
On the boot of the Appliances (either HCM92 and FSCM92), you may see several errors such as (check file in /var/log/messages)
Apr 27 10:31:33 localhost kernel: type=1400 audit(1367073093.328:11): avc:  denied  { write } for  pid=3437 comm="smbd" name="log" dev=sda2 ino=31163 scontext=system_u:system_r:smbd_t:s0 tcontext=system_u:object_r:var_log_t:s0 tclass=dir
Apr 27 10:31:33 localhost kernel: type=1400 audit(1367073093.328:12): avc:  denied  { add_name } for  pid=3437 comm="smbd" name="samba.log.192.168.1.204" scontext=system_u:system_r:smbd_t:s0 tcontext=system_u:object_r:var_log_t:s0 tclass=dir
Apr 27 10:31:33 localhost kernel: type=1400 audit(1367073093.328:13): avc:  denied  { create } for  pid=3437 comm="smbd" name="samba.log.192.168.1.204" scontext=system_u:system_r:smbd_t:s0 tcontext=system_u:object_r:var_log_t:s0 tclass=file
OVA_ESXi_050 
And also
OVA_ESXi_066
Whether they can be ignored, it can be solved later on, as following (I explained here some times ago):
Edit the file /etc/selinux/config, you’ll see
OVA_ESXi_064
For some reason, SELINUX variable is not set, give it the value “disabled” and reboot, no more messages should come out. 
OVA_ESXi_067
After rebooting, the error is gone
OVA_ESXi_068 

2) Report node definition
Unlike HCM92, on FSCM92, the report node is not defined:
OVA_ESXi_053
You should add it manually:
OVA_ESXi_056
Of course, now this report node must be linked to the server definition (server in use PRCS2951) :
OVA_ESXi_060 
And do not forget to give a password to the local node definition:
OVA_ESXi_062
Restart the Process Scheduler, and that’s it.
OVA_ESXi_058
Reports are now posted:
OVA_ESXi_063

3) Samba 
There are several samba errors such as (/var/log/messages):
Apr 27 14:09:52 hcm92000 smbd[3729]: [2013/04/27 14:09:52, 0] printing/print_cups.c:cups_connect(69)
Apr 27 14:09:52 hcm92000 smbd[3729]:   Unable to connect to CUPS server localhost:631 - Connection refused
Apr 27 14:09:52 hcm92000 smbd[3729]: [2013/04/27 14:09:52, 0] printing/print_cups.c:cups_connect(69)
Apr 27 14:09:52 hcm92000 smbd[3729]:   Unable to connect to CUPS server localhost:631 - Connection refused 
It comes from the Samba compilation options for printer sharing. Most likely you don’t want it. To fix it, connect as root, then modify the file/etc/samba/smb.conf, add the line “printing = bsd” :
[root@fscm92000 samba]# pwd
/etc/samba
[root@fscm92000 samba]#
[root@fscm92000 samba]# more smb.conf
[global]
workgroup = WORKGROUP
netbios name = VBOX
security = user
map to guest = Bad User
username map = /etc/samba/smbusers
guest account = nobody
guest ok = yes
log file = /var/log/samba.log.%m
log level = 2
max log size = 1000
socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192
printing = bsd
[client]
path = /opt/oracle/psft/pt/tools/setup/Client
writeable = no
available = yes
[ca]
path = /opt/oracle/psft/pt/tools/setup/PsCA
writeable = no
available = yes
[oracle-client]
path = /opt/oracle/psft/pt/oracle-client/windows
writeable = no
available = yes
[root@fscm92000 samba]#
The error shoud gone on next reboot (and/or next samba service start, see below).

4) Samba
 (bis)
You also may see several of the following errors (/var/log/messages):
Apr 27 13:16:57 fscm92000 nmbd[3798]: [2013/04/27 13:16:57, 0] nmbd/nmbd_nameregister.c:register_name_response(130)
Apr 27 13:16:57 fscm92000 nmbd[3798]:   register_name_response: server at IP 192.168.1.20 rejected our name registration of VBOX<20> IP 192.168.1.21 with error code 6.
Apr 27 13:16:57 fscm92000 nmbd[3798]: [2013/04/27 13:16:57, 0] nmbd/nmbd_mynames.c:my_name_register_failed(36)
Apr 27 13:16:57 fscm92000 nmbd[3798]:   my_name_register_failed: Failed to register my name VBOX<20> on subnet 192.168.1.21.
Apr 27 13:16:57 fscm92000 nmbd[3798]: [2013/04/27 13:16:57, 0] nmbd/nmbd_namelistdb.c:standard_fail_register(304)
Apr 27 13:16:57 fscm92000 nmbd[3798]:   standard_fail_register: Failed to register/refresh name VBOX<20> on subnet 192.168.1.21
I notice these errors only when I started HCM (here the IP=192.168.1.20) and FSCM (here IP=192.168.1.21) in the same time. If I start only one of them, then no errors appear.
It all comes because both have the same registration name – here VBOX – addressed to the DNS server, obviously the DNS does not like it. 
So, go to /etc/samba, edit smb.conf file and replace the “netbios name” parameter to something unique across your VMs (or network). For instance :
From
[global]
workgroup = WORKGROUP
netbios name = VBOX

To
[global]
workgroup = WORKGROUP
netbios name = FSCM92000

Restart the Samba service as following :
[root@hcm92000 samba]# /etc/rc.d/init.d/smb restart
Shutting down SMB services:                                [  OK  ]
Shutting down NMB services:                                [  OK  ]
Starting SMB services:                                     [  OK  ]
Starting NMB services:                                     [  OK  ]
[root@hcm92000 samba]#
Error is gone.

5) Auto-start
Last but not least, on the system reboot of FSCM92, whether the database is automatically restarted, the AppServer, Process Scheduler and WebServer are not (it is working on HCM92000).
To make sure they all are started on server boot, do the following (as root):
[root@fscm92000 ~]# cd /etc/init.d
[root@fscm92000 init.d]# ln -s /opt/oracle/psft/vm/appbatch-start .
[root@fscm92000 init.d]# cd ../rc3.d
[root@fscm92000 rc3.d]# ln -s ../init.d/appbatch-start S98appbatch-start
[root@fscm92000 rc3.d]# cd ../rc5.d
[root@fscm92000 rc5.d]# ln -s ../init.d/appbatch-start S98appbatch-start
It should make the trick on next server start.

6) Documentation
Very minor, on page 23 of the documentation provided along the VMs, it is mentioned the Peopletools client is in the folder /opt/oracle/psft/pt/tools/setup/client. But it does not work:
[root@fscm92000 ~]# cd /opt/oracle/psft/pt/tools/setup/client
-bash: cd: /opt/oracle/psft/pt/tools/setup/client: No such file or directory
[root@fscm92000 ~]#
The folder is /opt/oracle/psft/pt/tools/setup/Client… (note the Client with C in upper case, yes, Linux is case sensitive).

Enjoy,

Nicolas.

Saturday, April 27, 2013

Scripting VirtualBox Appliance move to VMWare ESXi

As an addendum of an article I wrote to explain how to move from the delivered VirtualBox Appliance for Peoplesoft to VMWare ESXi, there were several manual steps to be done in order to get it works.

It can be tedious to make some changes manually in the ovf file (see the details in the given link above). We all know that manual steps are subject to human error. Moreover, make one script may save your time, launch it and you can enjoy something else instead of waiting for the unzip to be completed, the untar, the push to ESXi host... And who like to do more than once the same tedious job ? 
Lastly, since Peoplesoft will deliver these VirtualBox Appliances every 8 weeks (!), this task has to be done quite often. So, no way but a script must be written somehow.

You can actually script every single step that I explained in the other blog entry (see the link above).
Even the download, within the wget option available on My Oracle Support, can be scripted, as shown below.
First accept the licence terms, and download the wget script.
OVA_ESXi_043OVA_ESXi_044 
This wget script is a shell script, you have to put your user’s password in it, and run it. There’s a timeout though, 8 hours to download all the 8 files (about 24.5Gb), depending of your bandwidth, it may or may not be enough. 
Mine is not that fast (sic), so I won’t use it for now and the following is assuming the zipped files are already there, previously downloaded by one or other way.

As I showed in the other blog entry, to move from VirtualBox Appliance to VMWare ESXi, I use the VMWare ovftool, so it is also assumed that you have it installed.

To be short, once you have the zipped files and the ovftool installed, you are ready to go.

As shown in the other blog post (see link above), there manual changes in the ovf file – and it can be confusing. Let’s use sed, that will change all at once.

I’ll use a parameter file replace.sed, and a script ova_to_esx.sh (see below).
Let’s say your are downloading FSCM92000 image, the zip files name format will be like FSCMDB-SES-85302d_ova_Xof8.zip.
The script takes one parameter as input, here it will be FSCMDB-SES-85302d. The destination VM name in VMWare ESXi will also be FSCMDB-SES-85302d.
All is in the same folder, zip files, replace.sed and ova_to_esx.sh.

So, here we go (test below done from a CentOS 6.4 VM hosted on my ESXi server):
[root@omsa FSCM92000]# ls -l
total 25118780
-rwxrw-rw- 1 root root        969 Apr 24 16:24 FSCMDB-SES-85302d_ova_1of8.zip
-rwxrw-rw- 1 root root 3703776794 Apr 24 19:34 FSCMDB-SES-85302d_ova_2of8.zip
-rwxrw-rw- 1 root root 3475881883 Apr 24 19:15 FSCMDB-SES-85302d_ova_3of8.zip
-rwxrw-rw- 1 root root 3721073992 Apr 24 19:25 FSCMDB-SES-85302d_ova_4of8.zip
-rwxrw-rw- 1 root root 3702449335 Apr 24 23:48 FSCMDB-SES-85302d_ova_5of8.zip
-rwxrw-rw- 1 root root 3730977958 Apr 25 01:09 FSCMDB-SES-85302d_ova_6of8.zip
-rwxrw-rw- 1 root root 3709152935 Apr 25 08:16 FSCMDB-SES-85302d_ova_7of8.zip
-rwxrw-rw- 1 root root 3653153461 Apr 25 20:12 FSCMDB-SES-85302d_ova_8of8.zip
-rwxr-xr-x 1 root root       1049 Apr 26 18:36 ova_to_esx.sh
-rwxr-xr-x 1 root root        938 Apr 24 18:00 replace.sed
[root@omsa FSCM92000]# more replace.sed ova_to_esx.sh
::::::::::::::
replace.sed
::::::::::::::
s/<OperatingSystemSection ovf:id="109">/<OperatingSystemSection ovf:id="101">/
s/<Description>Oracle_64<\/Description>/<Description>oracleLinux64Guest<\/Description>/
s/<vbox:OSType ovf:required="false">Oracle_64<\/vbox:OSType>/<vbox:OSType ovf:required="false">oracleLinux64Guest<\/vbox:OSType>/
s/<vssd:VirtualSystemType>virtualbox-2.2<\/vssd:VirtualSystemType>/<vssd:VirtualSystemType>vmx-07<\/vssd:VirtualSystemType>/
s/<rasd:Caption>sataController0<\/rasd:Caption>/<rasd:Caption>SCSI Controller0<\/rasd:Caption>/
s/<rasd:Description>SATA Controller<\/rasd:Description>/<rasd:Description>SCSI Controller<\/rasd:Description>/
s/<rasd:ElementName>sataController0<\/rasd:ElementName>/<rasd:ElementName>SCSI Controller0<\/rasd:ElementName>/
s/<rasd:ResourceSubType>AHCI<\/rasd:ResourceSubType>/<rasd:ResourceSubType>lsilogic<\/rasd:ResourceSubType>/
s/<rasd:ResourceType>20<\/rasd:ResourceType>/<rasd:ResourceType>6<\/rasd:ResourceType>/
::::::::::::::
ova_to_esx.sh
::::::::::::::
myVM=$1

echo `date`
echo "Start"
#Unzip the input zipfiles
for zipfile in `ls ${myVM}*.zip`
do
echo `date`
echo "unzipping file "$zipfile
unzip $zipfile
done

#Concatenation to one ova file
rm -f ${myVM}.ova
for ovafile in `ls ${myVM}.ova_?of?`
do
echo `date`
echo "concatenation file "$ovafile
cat $ovafile >> ${myVM}.ova
done

#Extract files from ova
echo `date`
echo "untar the ova file"
tar xvf ${myVM}.ova

#Backup the ovf
mv ${myVM}.ovf ${myVM}.ovf.orig

#Replace the string to be replace as defined in replace.sed file
sed -f replace.sed < ${myVM}.ovf.orig > ${myVM}.tmp

#Take the item line corresponding to the sound item group
line=`grep -ni -B7 sound ${myVM}.tmp|grep  "<Item>"|awk -F- '{print $1}'`
#It will replace the <Item> for the sound card item to be deactivated
sed -e "${line}s/<Item>/<Item ovf:required=\"false\">/" < ${myVM}.tmp > ${myVM}.ovf
rm -f ${myVM}.tmp

echo `date`
echo "Moving to ESXi"
ovftool --lax -ds=vm "--net:HostOnly=VM Network" ${myVM}.ovf "vi://root:adam2008@192.168.1.10:443"
echo `date`
echo "End"
[root@omsa FSCM92000]#
[root@omsa FSCM92000]# ./ova_to_esx.sh FSCMDB-SES-85302d
Fri Apr 26 18:36:56 CEST 2013
Start
Fri Apr 26 18:36:56 CEST 2013
unzipping file FSCMDB-SES-85302d_ova_1of8.zip
Archive:  FSCMDB-SES-85302d_ova_1of8.zip
  inflating: 16660432Readme.html
Fri Apr 26 18:36:56 CEST 2013
unzipping file FSCMDB-SES-85302d_ova_2of8.zip
Archive:  FSCMDB-SES-85302d_ova_2of8.zip
  inflating: FSCMDB-SES-85302d.ova_1of7
Fri Apr 26 18:45:11 CEST 2013
unzipping file FSCMDB-SES-85302d_ova_3of8.zip
Archive:  FSCMDB-SES-85302d_ova_3of8.zip
  inflating: FSCMDB-SES-85302d.ova_2of7
Fri Apr 26 18:53:24 CEST 2013
unzipping file FSCMDB-SES-85302d_ova_4of8.zip
Archive:  FSCMDB-SES-85302d_ova_4of8.zip
  inflating: FSCMDB-SES-85302d.ova_3of7
Fri Apr 26 19:01:57 CEST 2013
unzipping file FSCMDB-SES-85302d_ova_5of8.zip
Archive:  FSCMDB-SES-85302d_ova_5of8.zip
  inflating: FSCMDB-SES-85302d.ova_4of7
Fri Apr 26 19:10:20 CEST 2013
unzipping file FSCMDB-SES-85302d_ova_6of8.zip
Archive:  FSCMDB-SES-85302d_ova_6of8.zip
  inflating: FSCMDB-SES-85302d.ova_5of7
Fri Apr 26 19:18:43 CEST 2013
unzipping file FSCMDB-SES-85302d_ova_7of8.zip
Archive:  FSCMDB-SES-85302d_ova_7of8.zip
  inflating: FSCMDB-SES-85302d.ova_6of7
Fri Apr 26 19:27:05 CEST 2013
unzipping file FSCMDB-SES-85302d_ova_8of8.zip
Archive:  FSCMDB-SES-85302d_ova_8of8.zip
  inflating: FSCMDB-SES-85302d.ova_7of7
Fri Apr 26 19:35:15 CEST 2013
concatenation file FSCMDB-SES-85302d.ova_1of7
Fri Apr 26 19:43:45 CEST 2013
concatenation file FSCMDB-SES-85302d.ova_2of7
Fri Apr 26 19:52:27 CEST 2013
concatenation file FSCMDB-SES-85302d.ova_3of7
Fri Apr 26 20:00:55 CEST 2013
concatenation file FSCMDB-SES-85302d.ova_4of7
Fri Apr 26 20:09:25 CEST 2013
concatenation file FSCMDB-SES-85302d.ova_5of7
Fri Apr 26 20:17:58 CEST 2013
concatenation file FSCMDB-SES-85302d.ova_6of7
Fri Apr 26 20:26:30 CEST 2013
concatenation file FSCMDB-SES-85302d.ova_7of7
Fri Apr 26 20:34:51 CEST 2013
untar the ova file
FSCMDB-SES-85302d.ovf
FSCMDB-SES-85302d-disk1.vmdk
FSCMDB-SES-85302d-disk2.vmdk
FSCMDB-SES-85302d-disk3.vmdk
FSCMDB-SES-85302d-disk4.vmdk
FSCMDB-SES-85302d-disk5.vmdk
Fri Apr 26 21:35:49 CEST 2013
Moving to ESXi
Opening OVF source: FSCMDB-SES-85302d.ovf
Opening VI target: vi://root@192.168.1.10:443/
Deploying to VI: vi://root@192.168.1.10:443/
Transfer Completed
Warning:
- No manifest file found.
- No manifest entry found for: 'FSCMDB-SES-85302d-disk1.vmdk'.
- No manifest entry found for: 'FSCMDB-SES-85302d-disk2.vmdk'.
- No manifest entry found for: 'FSCMDB-SES-85302d-disk3.vmdk'.
- No manifest entry found for: 'FSCMDB-SES-85302d-disk4.vmdk'.
- No manifest entry found for: 'FSCMDB-SES-85302d-disk5.vmdk'.
Completed successfully
Fri Apr 26 21:59:11 CEST 2013
End
[root@omsa FSCM92000]#
Again, the manifest warning may be ignored.
Actually, took approximately  1 hour to unzip all the files, 1 hour to concatenate them all in one, 1 hour to extract and 20 minutes to move to ESXi.

And now, from vSphere Client, you can see the new virtual machine (I added already the CD/DVD drive for further VMWare tools installation): 
 OVA_ESXi_046
You can see above the HD1 is an IDE, whether the HD2,3,4 and 5 are SCSI (below). No idea why HD1 must be IDE.OVA_ESXi_047
As of now, you can start the VM and follow the instructions given by the Oracle doc.
OVA_ESXi_048
OVA_ESXi_049

Obviously, it’s just an example of script, everyone can adjust, add error handler or more parameters…

Enjoy,

Nicolas.

Wednesday, April 24, 2013

VirtualBox Appliances for PeopleSoft on VMWare ESXi

Last week, Oracle delivered new Peoplesoft demo images (available on My Oracle Suport only), please see the note PeopleSoft Update Manager Home Page [ID 1464619.1]

A lot of effort have been made here, that's really nice. It will greatly simplify the updates (PUM).

So far it was delivered as templates for Oracle VM. However, it made a lot of people crazy finding ways to make it running on their laptop for their own purpose. Jim Marion wrote a nice step-by-step here to to move from Oracle VM to VirtualBox.

Right now, it is all running on VirtualBox. Many people will be very happy - for personal use, Hakan Biroglu has already showed us how quickly it can be set, up and running here.

But now it is the other way around, what about people having bare-metal virtualization software only ?
As clearly stated in the documentation, it cannot run from VirtualBox installed onto a virutal OS (virtualization of a 64-bit OS on a virtualized environment does not work).

So, you must first have to have a physical machine strong enough (most likely 6Gb+ RAM) to get it running. Whether I can understand such a mechanism for a personal use of these VMs, I don't find it really usable if you have to share it with your developers. Moreover, what if you ever want to test messaging between two of the delivered Peoplesoft images (say FSCM and HCM) ?

If like me you don't have such hardware, you are stuck.
Or maybe you have bare-metal virtualization software on dedicated server host ? That would be great to make it work there.

On my side, I have ESXi vSphere Hypervisor 5.1 (build 1021289 – March 2013) running on my Dell Poweredge 2900III (1 quad-core CPU, 16Gb RAM), so let's have a try.
Whether it's not a direct operation, it does not look so complicated.

Note that all the steps below are done from within a CentOS 6.4 VM hosted on my ESX server.
1. Download the 8 zip files
Go to the note mentioned above, and download the module you want to work on, here I’m working on HCM9.2.

2. Unzip all the zip files
Regular unzip command to be used against every single downloaded files.

3. ova files concatenation
From the unzipped files, you'll get one html doc and 7 part of ova files to be concatenated as follow
cat HCMDB-SES-85302d.ova_1of7 \
HCMDB-SES-85302d.ova_2of7 \
HCMDB-SES-85302d.ova_3of7 \
HCMDB-SES-85302d.ova_4of7 \
HCMDB-SES-85302d.ova_5of7 \
HCMDB-SES-85302d.ova_6of7 \
HCMDB-SES-85302d.ova_7of7 > HCMDB-SES-85302d.ova
At this stage, you can delete the zipfiles as well as the 7 files HCMDB-SES-85302d.ova_?of7, it will save a lot of space.
So far, so good, it is all documented (see the pdf file "Using Oracle VM VirtualBox Appliances for PeopleSoft Applications" from the note mentioned above).

4. Extract ova file
Now, if using VirtualBox, you are normally almost done, the Oracle doc says all about VirtualBox, you may want to follow it.
Since we are using the bare-metal virtualization software VMWare ESXi, we will deviate a little bit from the official doc though.

Whether there is a way to go directly from ova file to ESX with the VMWare utility ovftool (here I used the version 3.0.1), it won't work straightaway. The ova file is coming from VirtualBox and it won't fit exactly to the ESX expectations.

Fortunately, the ova file is nothing but an archive file containing the vmdk files (the disks) and one ovf file (virtual machine definition settings, this is an xml file).

The ovf file may also be loaded in ESX through VMWare ovftool. This is this ovf file which must be modified to be ESX compliant and to be loaded into ESX.
First, untar the ova :
tar xvf HCMDB-SES-85302d.ova

5. Checking files
After all these steps, we have the following in the directory :
root@omsa HCM92000]# ls -l
total 100521488
-rw-r--r-- 1 root root        2530 Apr 13 22:05 16660429Readme.html
-rwxr-xr-x 1 root root         436 Apr 20 16:05 extract.sh
-rw------- 1 root root   609456128 Apr  1 08:32 HCMDB-SES-85302d-disk1.vmdk         <-- coming from ova extract
-rw------- 1 root root  3626726912 Apr  1 08:36 HCMDB-SES-85302d-disk2.vmdk         <-- coming from ova extract
-rw------- 1 root root 
5143003648 Apr  1 08:44 HCMDB-SES-85302d-disk3.vmdk         <-- coming from ova extract
-rw------- 1 root root 
7147702784 Apr  1 08:52 HCMDB-SES-85302d-disk4.vmdk         <-- coming from ova extract
-rw------- 1 root root 
9284810240 Apr  1 09:01 HCMDB-SES-85302d-disk5.vmdk         <-- coming from ova extract
-rw-r--r-- 1 root root 25811718656 Apr 20 18:03 HCMDB-SES-85302d.ova
-rw-r--r-- 1 root root  3758096384 Apr  1 10:12 HCMDB-SES-85302d.ova_1of7
-rwxrw-rw- 1 root root         967 Apr 19 19:27 HCMDB-SES-85302d_ova_1of8.zip
-rw-r--r-- 1 root root  3758096384 Apr  1 10:12 HCMDB-SES-85302d.ova_2of7
-rwxrw-rw- 1 root root  3702657177 Apr 20 00:44 HCMDB-SES-85302d_ova_2of8.zip
-rw-r--r-- 1 root root  3758096384 Apr  1 10:13 HCMDB-SES-85302d.ova_3of7
-rwxrw-rw- 1 root root 
3605212087 Apr 20 00:38 HCMDB-SES-85302d_ova_3of8.zip
-rw-r--r-- 1 root root  3758096384 Apr  1 10:13 HCMDB-SES-85302d.ova_4of7
-rwxrw-rw- 1 root root  3709723401 Apr 20 00:41 HCMDB-SES-85302d_ova_4of8.zip
-rw-r--r-- 1 root root  3758096384 Apr  1 10:14 HCMDB-SES-85302d.ova_5of7
-rwxrw-rw- 1 root root  3699819939 Apr 20 00:43 HCMDB-SES-85302d_ova_5of8.zip
-rw-r--r-- 1 root root  3758096384 Apr  1 10:14 HCMDB-SES-85302d.ova_6of7
-rwxrw-rw- 1 root root  3731137895 Apr 20 13:47 HCMDB-SES-85302d_ova_6of8.zip
-rw-r--r-- 1 root root  3263140352 Apr  1 10:15 HCMDB-SES-85302d.ova_7of7
-rwxrw-rw- 1 root root  3709400712 Apr 20 13:32 HCMDB-SES-85302d_ova_7of8.zip
-rwxrw-rw- 1 root root 
3239213691 Apr 20 13:59 HCMDB-SES-85302d_ova_8of8.zip
-rw------- 1 root root       15513 Apr  1 08:31 HCMDB-SES-85302d.ovf                 <-- coming from ova extract
-rwxrw-rw- 1 root root     1042477 Apr 20 11:20 Using_Oracle_VM_VirtualBox_Appliances_for_PeopleSoft_Applications.pdf
[root@omsa HCM92000]#

6. Edit the ovf file
Don't forget to take a backup of the existing file, it can save you
[root@omsa HCM92000]# cp HCMDB-SES-85302d.ovf HCMDB-SES-85302d.ovf.orig
Only few things have to changed in the ovf file to be ESX compliant. Here we go :
6.1) The operating system: ESX does not understand the OS definition name and id coming from VirtualBox, you may want to change it (otherwise you'll get a warning)
Change from (line 32, operating system id)
<     <OperatingSystemSection ovf:id="109">
to
>     <OperatingSystemSection ovf:id="101">
Change from (lines 34 and 35)
<       <Description>Oracle_64</Description>
<       <vbox:OSType ovf:required="false">Oracle_64</vbox:OSType>
to
>       <Description>oracleLinux64Guest</Description>
>       <vbox:OSType ovf:required="false">oracleLinux64Guest</vbox:OSType>
6.2) The virtual system type is obviously different, you must change it (otherwise it error out)
Change from (line 43)
<         <vssd:VirtualSystemType>virtualbox-2.2</vssd:VirtualSystemType>
to (here for ESX 5.1, it could be vmx-08 - VM tool version compatibility you define when creating a new VM, but did not tested it yet)
>         <vssd:VirtualSystemType>vmx-07</vssd:VirtualSystemType>
6.3 The SATA controller does not make ESX very happy, you must change it to SCSI (otherwise it errored out)
Change from (lines 82, 83 and 84)
<         <rasd:Caption>sataController0</rasd:Caption>
<         <rasd:Description>SATA Controller</rasd:Description>
<         <rasd:ElementName>sataController0</rasd:ElementName>
to
>         <rasd:Caption>SCSI Controller0</rasd:Caption>
>         <rasd:Description>SCSI Controller</rasd:Description>
>         <rasd:ElementName>SCSI Controller0</rasd:ElementName>
6.4) About sata and SCSI again, the subtype and type must be changed as well
Change from (lines 86 and 87)
<         <rasd:ResourceSubType>AHCI</rasd:ResourceSubType>
<         <rasd:ResourceType>20</rasd:ResourceType>
to
>         <rasd:ResourceSubType>lsilogic</rasd:ResourceSubType>
>         <rasd:ResourceType>6</rasd:ResourceType>
6.5) Lastly, ESX does not like sound card, it must be deactivated
Change from (line 106, check for the item corresponding to the sound card)
<       <Item>
to
>       <Item ovf:required="false">

7. From ovf to ESX

Now you are done with the modifications, next step is a move to ESX with ovftool which will read this ovf file and will directly create the VM on remote ESX pushing all the disks as well.
You must install the ovftool in order to continue:
[root@omsa OvfTool]# ./VMware-ovftool-3.0.1-801290-lin.x86_64.bundle
You must accept the VMware OVF Tool component for Linux End User
License Agreement to continue.  Press Enter to proceed.
VMWARE
OPEN VIRTUALIZATION FORMAT TOOL LICENSE AGREEMENT
...
Do you agree? [yes/no]: yes
The product is ready to be installed.  Press Enter to begin
installation or Ctrl-C to cancel.
Installing VMware OVF Tool component for Linux 3.0.1
    Configuring...
[######################################################################] 100%
Installation was successful.
[root@omsa OvfTool]#
Then you can go ahead with the command line. Here below :
-ds is the datastore you want to push the VM to (here it is vm)
--net is the network definition in your ESX host, the input ovf has HostOnly, ESX does not know about it, just specify your own (here it is VM Network)
replace <mypassword> by your ESX host user password
replace the IP address by the IP address (or hostname) of the ESX server
[root@omsa HCM92000]# ovftool --lax -ds=vm "--net:HostOnly=VM Network" HCMDB-SES-85302d.ovf "vi://root:<mypassword>@192.168.1.10:443"
Opening OVF source: HCMDB-SES-85302d.ovf
Opening VI target: vi://
root@192.168.1.10:443/
Deploying to VI: vi://root@192.168.1.10:443/
Transfer Completed
Warning:
- No manifest file found.
- No manifest entry found for: 'HCMDB-SES-85302d-disk1.vmdk'.
- No manifest entry found for: 'HCMDB-SES-85302d-disk2.vmdk'.
- No manifest entry found for: 'HCMDB-SES-85302d-disk3.vmdk'.
- No manifest entry found for: 'HCMDB-SES-85302d-disk4.vmdk'.
- No manifest entry found for: 'HCMDB-SES-85302d-disk5.vmdk'.
Completed successfully
[root@omsa HCM92000]#
Never mind the "manifest" warning, it does not matter.
8. Start and configure the VM
From within vSphere Client, start the VM.
OVA_ESXi_002
Here below the settings of the VM:
OVA_ESXi_006 
As of now, you fall back to the Oracle standard procedure and can follow the doc. Before hands, you may or may not want to have SES, keep 4Gb RAM or change it to 6Gb respectively.
Bear in mind that you probably don't want DHCP configuration to be able to easily share the VM with your colleagues.
OVA_ESXi_003
OVA_ESXi_008
Specify the root password and DHCP=n (to have a fixed IP address)
OVA_ESXi_010
OVA_ESXi_013
The database will start immediately as well as AppServer, Batch server, and PIA, specify if you want SES or not
OVA_ESXi_015
The you get the prompt
OVA_ESXi_016
And it’s done.
Adjust your DNS or local hosts file, and connect to the PIA (I’m on Windows XP, that’s why the warning):
OVA_ESXi_017
Note the domain name is mandatory in the url.
OVA_ESXi_018
Report node is already defined
OVA_ESXi_019
However, for some reason the distribution is blank in the server definition, you  have to specify it
OVA_ESXi_021
Now, after restarting the batch server, you can run batches and check the output
OVA_ESXi_023
OVA_ESXi_025
Note that unlike the Peoplesoft templates on Oracle VM, there’s no COBOL configured.
 
9. VMWare tools
You may want to install the VMWare tools afterwards.
First, stop the VM and add a CD-ROM
OVA_ESXi_034
OVA_ESXi_036  
Restart the VM, and install the tools
OVA_ESXi_030
OVA_ESXi_038
OVA_ESXi_039
And finally we get
OVA_ESXi_040   

By the way, speaking of running demo on bare-metal VM, do these images build on VirtualBox make the end of the Peoplesoft VM templates for Oracle VM ?

Enjoy,

Nicolas.