Create domain user and explicitly declare it
as a member of local administrator group on each cluster node as per Oracle
recommendation “Oracle strongly recommends the use of domain users for Oracle
RAC installations on Windows platforms.”
1. Change the elevation prompt behavior
for administrators to "Elevate without prompting" to allow for user equivalence to function properly in Windows 2008:
·
Open
a command prompt and type "secpol.msc" to launch the Security
Policy Console management utility.
·
From
the Local Security Settings console tree, click Local Policies, and then
Security Options Scroll down to and double-click User Account Control: Behavior
of the elevation prompt for administrators.
·
From
the drop-down menu, select: "Elevate without prompting (tasks requesting
elevation will automatically run as elevated without prompting the
administrator)"
·
Click
OK to confirm the changes.
·
Repeat
the above steps on all cluster nodes.
2. Ensure that the Administrators group
is listed under "Manage auditing and security log":
·
Open
a command prompt and type "secpol.msc" to launch the Security
Policy Console management utility.
·
Click
on "Local Policies"
·
Click
on "User Rights Assignment"
·
Locate
and double click the "Manage auditing and security log" in the
listing of User Rights Assignments.
·
If
the Administrators group is NOT listed in the "Local Security
Settings" tab, add the group now.
·
Click
OK to save the changes (if changes were made).
·
Repeat
the above steps on all cluster nodes.
When installing Oracle Grid
Infrastructure and/or Oracle RAC it is required to turn off the Windows
firewall. Follow these steps to turn off the windows firewall:
·
Click
Start, click Run, type "firewall.cpl", and then click OK
·
In
the Firewall Control Panel, click "Turn Windows Firewall on or off"
(upper left hand corner of the window).
·
Choose
the "Off" radio button in the "Windows Firewall Settings"
window and click OK to save the changes.
·
Repeat
the above steps on cluster nodes.
OR
Execute the below command:
netsh
advfirewall set Allprofiles state off
Disable Antivirus running on cluster
nodes before and for the entire duration of the installation of Oracle on
Windows (be mindful of manual reboots during this time). Anti-Virus software
may of course be re-enabled following the installation but the following should
be excluded from being scanned:
·
Oracle
Software directories
·
OCFS
file systems
·
ACFS
file systems
·
Network
scanning of the private interconnect.
Desktop heap size is required to be
increased in the registry for the database server. It can be found in the
following location:
\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session
Manager\SubSystems\
·
Edit
the Windows value of Type REG_EXPAND_SZ.
It will look similar to:
%SystemRoot%\system32\csrss.exe
ObjectDirectory=\Windows SharedSection=1024,3072,512 Windows=On
SubSystemType=Windows ServerDll=basesrv,1
ServerDll=winsrv:UserServerDllInitialization,3 ServerDll=winsrv:ConServerDllInitialization,2
ProfileControl=Off MaxRequestThreads=16
·
At
this location the following set of values can be seen for
"SharedSection", 1024, 3072, 512. The third value (512) is the size
of the desktop heap for each desktop that is associated with a
"noninteractive" window station.
·
Increase
of the third value to 1024, so the values are now listed as 1024, 3072, 1024
resolved the problem.
·
If
this value is not present, the size of the desktop heap for noninteractive
window stations will be same as the size specified for interactive window
stations (the second SharedSection value).
Oracle recommends that the paging file must be
size to at least one and a half times the amount of RAM for systems with 4 to
32 GB of RAM. For systems with greater than 32 GB of RAM, Oracle recommends
that the paging file must be set to 32 GB.
To set the TEMP and TMP environment
variables: (TMP=C:\Windows\temp TEMP= C:\Windows\temp)
·
Log
into the server as the user that will perform the installation
·
Open
Computer Properties
·
Click
the Advanced system settings link (on the left under tasks)
·
Under
the Advanced tab, click the Environment Variables button
·
Modify
the TEMP and TMP variables under "User variables for Administrator"
to the desired setting.
·
Keep
in mind, this path must be identical for both TMP and TEMP and they must be set
to the same location on all cluster nodes.
·
Click
OK to save the changes.
·
Repeat
the above steps for all cluster nodes.
For high availability and load balancing it is
recommended to create network card bonding for public adapters. With the
assistance of system administrator two network teaming were created on both
nodes one for public network and another for private network.
Using HP virtual connect flex fabric a private
interconnect is configured by the system admin. For the private network, the
end points of all designated interconnect interfaces must be completely
reachable on the network. Every node in the cluster should be able to connect
to every private network interface in the cluster.
The IP Addresses are allocated for each cluster node
respectively
Identity
|
Host Node
|
Name
|
Type
|
Address
|
Resolved by
|
Node1 Public
|
psrv1
|
psrv1
|
Public
|
162.30.0.150
|
DNS
|
Node1 VIP
|
psrv1
|
psrv1-vip
|
Virtual
|
162.30.0.156
|
DNS, Host file
|
Node1 Private
|
psrv1
|
psrv1-priv
|
Private
|
192.168.188.1
|
Host file
|
Node2 Public
|
psrv2
|
psrv2
|
Public
|
162.30.0.151
|
DNS
|
Node2 VIP
|
psrv2
|
psrv2-vip
|
Virtual
|
162.30.0.157
|
DNS, Host file
|
Node2 Private
|
psrv2
|
psrv2-priv
|
Private
|
192.168.188.2
|
Host file
|
|
SCAN VIP1
|
None
|
psrv-scan
|
Virtual
|
162.30.0.158
|
DNS
|
SCAN VIP2
|
None
|
psrv-scan
|
Virtual
|
162.30.0.159
|
DNS
|
SCAN VIP3
|
None
|
psrv-scan
|
Virtual
|
162.30.0.160
|
DNS
|
#PublicLAN
162.30.0.150 psrv1.feco.com psrv1
162.30.0.151 psrv2.feco.com psrv2
#VIP
162.30.0.156 psrv1-vip.feco.com
psrv1-vip
162.30.0.157 psrv2-vip.feco.com
psrv2-vip
#PrivateLAN
192.168.188.1 psrv1-priv
192.168.188.2 psrv2-priv
Run the following command with admin
user on each cluster node:
C:\Users\Administrator>
netsh interface ipv4 set global dhcpmediasense=disabled
C:\Users\Administrator>
netsh interface ipv6 set global dhcpmediasense=disabled
Validate the change with: C:\Users\Administrator>
netsh interface ipv4 show global
OR
Disable Media Sensing by completing
the following steps on each node of your cluster:
·
Use
Registry Editor (Regedit.exe) to view the following key in the registry:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
·
Add
the following registry entry to the Parameters subkey:
Name:
DisableDHCPMediaSense
Data
type: REG_DWORD (Boolean)
Value:
1
·
Exit
the Registry Editor and restart the computer.
Run the following command with admin
user on each cluster node:
C:\Users\Administrator>
netsh int tcp set global chimney=disabled
C:\Users\Administrator>
netsh int tcp set global rss=disabled
Validate these changes with: C:\Users\Administrator> netsh interface
tcp show global
Perform the follow tasks to ensure
this requirement is met:
·
Click
Start, click Run, type "ncpa.cpl", and then click OK.
·
In
the menu bar on the top of the window click "Advanced" and choose
"Advanced Settings" (For Windows 2008, if the "Advanced" is
not showing, click 'Alt' to enable that menu item).
·
Under
the Adapters and Bindings tab use the up arrow to move the Public interface to
the top of the Connections list.
·
Under
Binding order for increase the priority of IPv4 over IPv6
·
Click
OK to save the changes
·
Repeat
the above steps on all cluster nodes.
Invoke Server Manager
·
select
'View Network Connections'
·
select
the 'public' network interface
·
choose
'Properties' from the right click menu
·
select
'Internet Protocol Version 4(TCP/IPv4)' from the 'Networking' tab
·
click
'Properties'
·
click
'Advanced...' from the 'General' tab
·
select
the 'DNS' tab
·
Unclick
the radio button for 'Register this connection's addresses in DNS'.
To disable IPv6 components follow the below steps:
·
Start
the regedit program. Create a backup of the current registry settings.
·
In
the Registry Editor, go to the
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters entry.
·
Double-click
the DisabledComponents key to modify its value. If the DisabledComponents key
does not exist, then create this key as follows:
·
Open
the Edit menu.
·
Select
New > DWORD (32-bit) Value.
·
Enter
DisabledComponents, and then press Enter to create the key.
·
For
the DisabledComponents key, add a DWORD value of 0xffffffff.
·
Exit
the Registry Editor and restart the computer.
·
Repeat
these steps on each node on which you plan to install Oracle Grid
Infrastructure.
·
These
steps disable all IPv6 components except for the IPv6 loopback interface.
There can be issues with some (non-Oracle)
services, which may already be running on the cluster nodes. Typically a
Microsoft Service: Distributed Transaction Coordinator (MSDTC) can interact
with Oracle software during install. It is recommended that this service is
stopped and set to 'manual' start using services.msc on both nodes.
It is recommended to create the stage for the
required software onto a local drive on Node1 (psrv1) of the cluster. The
following software is staged on local disk (C:\Oracle11gR2\Stage).
Grid Infrastructure (clusterware and ASM):
Oracle Database 11g Release 2 Grid Infrastructure
(11.2.0.3.0) for Windows
RDBMS software:
Oracle Database 11g Release 2 (11.2.0.3.0) for
Windows
To configure Windows Time service, use the
following command on each node:
C:\> W32tm
/register
To modify the Windows Time service to work in an
Oracle RAC environment, perform the following steps:
·
Open
the Registry Editor (regedit).
·
Locate
the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config key.
·
Set the following Windows Time service
parameters to these values:
·
MaxPosPhaseCorrection
to 600
·
MaxNegPhaseCorrection
to 600
·
MaxAllowedPhaseOffset
to 600
·
These
parameter settings specify that small time adjustments are allowed when the
time difference between the reference and cluster nodes is less than 10
minutes.
·
To
put the changes into effect, use the following command:
C:\>W32tm
/config /update
As per the recommendation of Oracle the
appropriate SAN MPIO drivers should be installed on the database servers in
order to achieve optimal performance on the storage LUNs allocated from the
SAN.
The "net use" utility can be used to
validate the ability to perform the software copy among the cluster nodes.
·
Open
a command prompt and execute the following
C:\Users\Administrator>net
use \\psrv1\C$, \\psrv2\C$
·
Repeat
the above steps on all cluster nodes.
The following storage LUNs are allocated to the
cluster nodes:
Disk
|
Description
|
Server
|
Drive
|
Size
|
Type
|
Disk 2
|
Oracle Software
|
psrv1
|
E:
|
100GB
|
Non-shared
|
Disk 2
|
Oracle Software
|
psrv2
|
E:
|
100GB
|
Non-shared
|
Disk 3
|
Oracle DATA
|
psrv1, psrv2
|
None
|
500GB
|
Shared
|
Disk 4
|
Oracle FRA
|
psrv1, psrv2
|
None
|
500GB
|
Shared
|
Disk 5
|
Oracle FRA
|
psrv1, psrv2
|
None
|
500GB
|
Shared
|
Disk 6
|
Oracle OCRVOTE
|
psrv1, psrv2
|
None
|
3GB
|
Shared
|
Disk 7
|
Oracle OCRVOTE
|
psrv1, psrv2
|
None
|
3GB
|
Shared
|
Disk 8
|
Oracle OCRVOTE
|
psrv1, psrv2
|
None
|
3GB
|
Shared
|
Enable auto mounting of disks in order to be visible to Oracle Grid
Infrastructure.
·
On
each node log on with Administrator privileges
·
Click
START->RUN and type diskpart.
C:\>diskpart
DISKPART>AUTOMOUNT
ENABLE
·
Repeat
the above command on all nodes in the cluster and restart the server.
The write caching must be disabled on all disks that will be used to share
data between the nodes in the cluster. Perform these steps to disable write
caching:
·
Click
Start and then select Control Panel, then Administrative Tools, then Computer
Management, then Device Manager, and then Disk
drives
·
Expand
the Disk drives and double-click the first drive listed.
·
Under
the Policies tab for the selected drive, uncheck the option that enables write caching.
·
Double-click
each of the other drives that will be used by Oracle cluster ware and
Oracle RAC and disable write caching as
described in the previous step.
Clean the shared disks before starting the installation. Cleaning will
remove data from any previous failed install. On Node1 from within diskpart you
should clean each of the disks. WARNING this will destroy all of the
data on the disk. Do not select the disk containing the operating system or you
will have to reinstall the OS. Cleaning the disk ‘scrubs’ every block on the
disk. This may take some time to complete.
·
Click
START->RUN and type diskpart.
C:\>diskpart
·
DISKPART>list
disk
·
The
entire available disks are listed with the above command. Select the shared
disk which is required for ASM disk groups and clean each disk by executing the
below command:
DISKPART>select disk 1
Disk 1 is
now the selected disk.
DISKPART> clean all
·
Repeat
the above step for all shared disk that would be used as a member of ASM disk
groups on one node only.
Assuming the disks are clean and completely empty, now create an extended
partition and then inside that partition a logical partition. Execute the below
commands to create logical partitions inside extended partitions:
·
Click
START->RUN and type diskpart.
C:\>diskpart
DISKPART> select disk 1
Disk 1 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified
partition.
DISKPART> create part log
DiskPart succeeded in creating the specified
partition.
·
Create
partitions for all shared disk to be used by ASM disk groups on one node only.
Diskpart should not add drive letters to the partitions on the local node.
The partitions on the other node may have drive letters assigned and it must be
removed as follows:
·
Click
START->RUN and type diskpart on node2.
C:\>diskpart
DISKPART> list volume
·
The
volumes are listed in a completely different order compared to the disk list.
Also against each volume the drive letter is assigned. Now remove the drive
letter using diskpart as shown below:
DISKPART>
select volume 1
Volume 1 is
the selected volume.
DISKPART>
remov
DiskPart successfully removed the drive letter
or mount point.
·
Ensure
that drive letter C and E are not removed, as C drive is a local disk and E
drive contains Oracle software.
The only partitions that the Oracle Universal Installer acknowledges on
Windows systems are logical drives that are created on top of extended
partitions and that have been stamped as candidate ASM disks. Therefore prior
to running the OUI the disks that are to be used by Oracle RAC MUST be stamped
using ASM Tool.
Perform following task to mark disk partitions for ASM:
·
Navigate
to the asmtool directory within the Grid Infrastructure installation media and
double click the asmtoolg.exe executable.
·
From
ASM graphical tool, select Add or Change Label and click Next.
·
On
the Select Disks screen choose the appropriate disks to be assigned a label and
enter an ASM Label Prefix to make the disks easily identifiable for their
intended purpose, size and/or performance characteristics. After choosing the
intended disks and entering the appropriate ASM Label Prefix, click Next
to continue.
·
Review
the summary screen and click Next.
·
On
the final screen, click Finish to update the ASM Disk Labels.
The following ASM Label prefix is marked for the disk partition:
Device
|
ASM Link Name
|
Size (GB)
|
Status
|
\Device\Hardisk3\Partition1
|
ORCLDISKDATA0
|
500
|
Stamped ASM device
|
\Device\Hardisk4\Partition1
|
ORCLDISKFRA0
|
500
|
Stamped ASM device
|
\Device\Hardisk5\Partition1
|
ORCLDISKFRA1
|
500
|
Stamped ASM device
|
\Device\Hardisk6\Partition1
|
ORCLDISKOCRVOTE0
|
3
|
Stamped ASM device
|
\Device\Hardisk7\Partition1
|
ORCLDISKOCRVOTE1
|
3
|
Stamped ASM device
|
\Device\Hardisk8\Partition1
|
ORCLDISKOCRVOTE2
|
3
|
Stamped ASM device
|
Prior to installing Grid Infrastructure it is
highly recommended to run the cluster verification utility (CLUVFY) to verify
that the cluster nodes have been properly configured for a successful Oracle
Grid Infrastructure installation. There are various levels at which CLUVFY can
be run, at this stage it should be run in the CRS pre-installation mode. Later
in this document CLUVFY will be run in pre dbinst mode to validate the
readiness for the RDBMS software installation.
·
Login
to the server in which the installation will be performed as the Local
Administrator.
·
Open
a command prompt and run CLUVFY as follows to perform the Oracle Cluster ware
pre-installation verification:
cmd> runcluvfy stage -post hwos -n [-verbose]
cmd> runcluvfy stage -pre crsinst -n
[-verbose]
·
On
the Setting up Grid Infrastructure Option Screen, choose “Skip software updates”
and click “Next” to continue.
· On the Select Installation Option Screen, choose “Install and Configure Grid Infrastructure for a Cluster” and click “Next” to continue.
· On the Grid Plug and Play Information screen enter “psrv-cluster” as desired Cluster Name, “psrv-scan.feco.com” as SCAN Name and use the default 1521 for SCAN Port. Uncheck the Configure GNS checkbox and Click “Next” to continue.
· By default OUI displays the information of the local node only, Click “Add” button and enter “psrv2.feco.com” as Public Hostname and “psrv2-vip.feco.com” as Virtual hostnames for the second node in the cluster.
· On the Specify Network Interface Usage screen, Ensure that public interface type is selected for 172.20.0.0 subnet and private for 192.168.188.0 subnet. Click “Next”to continue.
· Choose “Automatic Storage Management (ASM)” on the Storage Option Information screen and click “Next” to create the ASM diskgroup for Grid Infrastructure.
· On the Create ASM Disk Group screen, Enter OCRVOTE as the Diskgroup Name for storing OCR and voting disks. Choose “External” for Redundancy and select ORCLDISKOCRVOTE0 as the candidate disk. Let the AU size be 1 MB for OCR & Voting Diskgroup. Click “Next” to continue.
· Enter the passwords as “*****” for SYS and “******” for ASMSNMP users without quotation marks on the Specify ASM Passwords screen and click “Next” to continue.
· On the Failure Isolation Support screen choose “Do not use Intelligent Platform Management Interface (IPMI)”and click “Next” to continue.
· Specify the location for Oracle Base (E:\app\oracle) and the Grid Infrastructure Installation (E:\app\11.2.0\grid) on the Specify Installation Location screen.
· Click “Next” to allow the OUI to perform the prerequisite checks on the target nodes.
· After the prerequisite checks have successfully completed, review the summary of the pending installation and click ”Install” to install and configure Grid Infrastructure.
· On the Install Product screen ensure that all the installation and configuration succeeded
· On the Finish screen Click “Close” button to exit the Oracle Universal Installer.
E:\app\11.2.0\grid\bin\crsctl stat res -t