Symantec Veritas 5 Instrukcja Użytkownika

Veritas Cluster ServerInstallation GuideLinux for IBM Power5.0 Release Update 3

Verifying LLT, GAB, and cluster operation ... 113Verifying LLT ...

3Deport the coordinator disk group:# vxdg deport vxfencoorddg4Import the disk group with the -t option to avoid automatically importing itwhen the nod

# cp /etc/vxfen.d/vxfenmode_scsi3_raw /etc/vxfenmode3To check the updated /etc/vxfenmode configuration, enter the followingcommand on one of the nodes

6Save and close the file.7Verify the syntax of the file /etc/VRTSvcs/conf/config/main.cf:# hacf -verify /etc/VRTSvcs/conf/config8Using rcp or another

To verify I/O fencing configuration◆On one of the nodes, type:# vxfenadm -dI/O Fencing Cluster Information:================================Fencing Pro

Configuring VCS clusters for data integritySetting up disk-based I/O fencing manually104

Verifying the VCSinstallationThis chapter includes the following topics: About verifying the VCS installation About the LLT and GAB configuration fi

The file llthosts is a database that contains one entry per system. This filelinks the LLT system ID (in the first column) with the LLT host name. Thi

Note: The use of the -c -x option for /sbin/gabconfig is not recommended.About the VCS configuration file main.cfThe VCS configuration file /etc/VRTSv

Refer to the Veritas Cluster Server User's Guide to review the configurationconcepts, and descriptions of main.cf and types.cf files for Linux fo

NIC csgnic (Device = eth0NetworkHosts = { "192.168.1.17", "192.168.1.18" })NotifierMngr ntfr (SnmpConsoles = { "saturn"

Bringing up the existing node ... 145Installing the VCS software manually when adding a node to asingle

SystemList = { galaxy = 0, nebula = 1 }Parallel = 1OnlineRetryLimit = 3OnlineRetryInterval = 120)Phantom phantom_vxss ()ProcessOnOnly vxatd (IgnoreArg

..In the following main.cf file example, bold text highlights global cluster specificentries.include "types.cf"cluster vcs03 (ClusterAddress

Device = eth0)NotifierMngr ntfr (SnmpConsoles = { vcslab4079 = SevereError }SmtpServer = "smtp.veritas.com"SmtpRecipients = { "johndoe@

PathName = "/opt/VRTSat/bin/vxatd")// resource dependency tree//// group VxSS// {// Phantom phantom_vxss// ProcessOnOnly vxatd// }Verifying

3Verify LLT operation.See “Verifying LLT” on page 114.4Verify GAB operation.See “Verifying GAB” on page 116.5Verify the cluster operation.See “Verifyi

5To view additional information about LLT, run the lltstat -nvv commandon each node.For example, run the following command on the node galaxy in a two

However, the output in the example shows different details for the nodenebula. The private network connection is possibly broken or the informationin

To verify GAB1To verify that GAB operates, type the following command on each node:/sbin/gabconfig -a2Review the output of the command: If GAB operat

To verify the cluster1To verify the status of the cluster, type the following command:hastatus -summaryThe output resembles:-- SYSTEM STATE-- System S

The example shows the output when the command is run on the node galaxy.The list continues with similar information for nebula (not shown) and anyothe

Contents12

ValueAttribute#SystemLimitsgalaxyeth1 UP eth2 UPLinkHbStatusgalaxy0LoadTimeCountergalaxy600LoadTimeThresholdgalaxy80LoadWarningLevelgalaxy0NoAutoDisab

Adding and removingcluster nodesThis chapter includes the following topics: About adding and removing nodes Adding a node to a cluster Removing a n

Table 7-1Tasks that are involved in adding a node to a cluster (continued)ReferenceTaskSee “Preparing for a manual installation when adding a node”on

Figure 7-1Adding a node to a two-node cluster using two switchesPublic networkPrivatenetworkNew node:saturnTo set up the hardware1Connect the VCS priv

See “Mounting the product disc” on page 49.To prepare for installation◆Depending on the OS distribution, replace the dist in the command with rhel5or

# rpm -i VRTScutil-5.0.33.00-RU3_GENERIC.noarch.rpm# rpm -i VRTSatClient-4.3.28.0-0.ppc.rpm# rpm -i VRTSatServer-4.3.28.0-0.ppc.rpm SLES10/ppc64, req

Setting up the node to run in secure modeYou must follow this procedure only if you are adding a node to a cluster that isrunning in secure mode. If y

# vssat deletecred --domain type:domainname \--prplname prplnameFor example:# vssat deletecred --domain vx:[email protected] \--prplname sa

4Configure AB on node saturn to talk to RB1.# vxatd -o -a -n prplname -p password -x vx -y domainname -q \rootbroker -z 2821 -h roothash_file_pathFor

If the file on one of the existing nodes resembles:0 galaxy1 nebula Update the file for all nodes, including the new one, resembling:0 galaxy1 nebu

Introducing Veritas ClusterServerThis chapter includes the following topics: About Veritas Cluster Server About VCS basics About VCS features Abou

The -n flag indicates to VCS the number of nodes that must be ready toform a cluster before VCS starts.2On the new node, run the command, to configure

3Stop VCS on the new node:# hastop -sys saturn4Copy the main.cf file from an existing node to your new node:# rcp /etc/VRTSvcs/conf/config/main.cf \sa

Table 7-3Tasks that are involved in removing a nodeReferenceTaskSee “Verifying the status of nodes andservice groups” on page 132. Back up the config

To verify the status of the nodes and the service groups1Make a backup copy of the current configuration file, main.cf.# cp -p /etc/VRTSvcs/conf/confi

To remove or switch service groups from the departing node1Switch failover service groups from the departing node. You can switch grp3from node saturn

6Delete the departing node from the SystemList of service groups grp3 andgrp4.# hagrp -modify grp3 SystemList -delete saturn# hagrp -modify grp4 Syste

Modifying configuration files on each remaining nodePerform the following tasks on each of the remaining nodes of the cluster.To modify the configurat

Unloading LLT and GAB and removing VCS on the departing nodePerform the tasks on the node that is departing the cluster.If you have configured VCS as

# rpm -e VRTSvlic# rpm -e VRTSperl# rpm -e VRTSpbx# rpm -e VRTSicsco# rpm -e VRTSatServer# rpm -e VRTSatClient5Remove the LLT and GAB configuration fi

Installing VCS on a singlenodeThis chapter includes the following topics: About installing VCS on a single node Creating a single-node cluster using

Figure 1-1 illustrates a typical VCS configuration of four nodes that are connectedto shared storage.Figure 1-1Example of a four-node VCS clusterClien

Table 8-1Tasks to create a single-node cluster using the installerReferenceTaskSee “Preparing for a single node installation” on page 140.Prepare for

Answer y if you plan to incorporate the single node cluster into a multi-nodecluster in the future.Continue with the installation.See “Licensing VCS”

See “Preparing for a manual installation when adding a node” on page 123. See “Installing VCS RPMs for a manual installation” on page 124. See “Ad

Adding a node to a single-node clusterAll nodes in the new cluster must run the same version of VCS. The exampleprocedure refers to the existing singl

Setting up a node to join the single-node clusterThe new node to join the existing single node running VCS must run the sameversion of operating syste

To install and configure Ethernet cards for private network1Shut down VCS on Node A.# hastop -local2Shut down the node to get to the OK prompt:# shutd

5Freeze the service groups.# hagrp -freeze group -persistentRepeat this command for each service group in step 4.6Make the configuration read-only.# h

It handles the following tasks: Traffic distribution Heartbeat trafficConfigured as described in the following sections.Setting up /etc/llthostsThe

Table 8-4LLT directivesDescriptionDirectiveAssigns the system ID or symbolic name. The system ID numbermust be unique for each system in the cluster,

Configuring GAB when adding a node to a single node clusterVCS uses the Group Membership Services/Atomic Broadcast (GAB) protocol forcluster membershi

Figure 1-2Two examples of shared storage configurationsFully shared storage Distributed shared storageAbout LLT and GABVCS uses two components, LLT an

To reconfigure VCS on existing nodes1On Node A, create the files /etc/llttab, /etc/llthosts, and /etc/gabtab. Use thefiles that are created on Node B

To verify the nodes' configuration1On Node B, check the cluster membership.# gabconfig -a2Start the VCS on Node B.# hastart3Verify that VCS is up

Installing VCS on a single nodeAdding a node to a single-node cluster152

Uninstalling VCSThis chapter includes the following topics: About the uninstallvcs program Preparing to uninstall VCS Uninstalling VCS 5.0 RU3About

If you have manually edited any of the VCS configuration files, you need toreformat them.Uninstalling VCS 5.0 RU3You must meet the following conditi

3Enter the names of the systems from which you want to uninstall VCS.The program performs system verification checks and asks to stop all runningVCS p

The uninstallvcs program is not available in /opt/VRTS/install.Uninstalling VCSUninstalling VCS 5.0 RU3156

Advanced VCS installationtopicsThis appendix includes the following topics: Using the UDP layer for LLT Performing automated VCS installations Inst

Make sure that the LLT private links are on different physical networks.If the LLT private links are not on different physical networks, then make s

See “Sample configuration: direct-attached links” on page 161. See “Sample configuration: links crossing IP routers” on page 163.Table A-1 describe

Figure 1-3 illustrates a two-node VCS cluster where the nodes galaxy and nebulahave two private network connections.Figure 1-3Two Ethernet connections

Table A-2Field description for set-addr command in /etc/llttabDescriptionFieldThe ID of the cluster node; for example, 0.node-idThe string that LLT us

For example, with the following interfaces: For first network interfaceIP address=192.168.30.1, Broadcast address=192.168.30.255,Netmask=255.255.255.

Figure A-1A typical configuration of direct-attached links that use LLT overUDPUDP Endpoint eth1;UDP Port = 50001;IP = 192.1.3.1;Link Tag = link2UDP E

link link1 udp - udp 50000 - 192.1.2.2 192.1.2.255link link2 udp - udp 50001 - 192.1.3.2 192.1.3.255Sample configuration: links crossing IP routersFig

set-addr 3 link1 192.1.7.3set-addr 3 link2 192.1.8.3#disable LLT broadcastsset-bcasthb 0set-arp 0The /etc/llttab file on Node 0 resembles:set-node Nod

To perform automated installation1Navigate to the folder containing the installvcs program.# cd /mnt/cdrom/cluster_server2Start the installation from

$CPI::CFG{CMC_SERVICE_PASSWORD}="U2FsdVkX18v...n0hTSWwodThc+rX";$CPI::CFG{ENCRYPTED}="U2FsdGVkX1+k2DHcnW7b6...ghdh+zW4G0WFIJA=";$C

Table A-3Response file variables (continued)DescriptionVariableList of systems on which the product is to be installed,uninstalled, or configured.List

Table A-3Response file variables (continued)DescriptionVariableDefines a location, typically an NFS mount, from whichall remote systems can install pr

Table A-3Response file variables (continued)DescriptionVariableList of keys to be registered on the system.List or scalar: listOptional or required: o

About VCS featuresYou can use the Veritas Installation Assessment Service to assess your setup forVCS installation.See “Veritas Installation Assessmen

Table A-3Response file variables (continued)DescriptionVariableDefines the minimum severity level of messages(Information, Warning, Error, SevereError

Table A-3Response file variables (continued)DescriptionVariableList of systems where VCS must be uninstalled.List or scalar: scalarOptional or require

5After the installation is complete, review the installer report.The installer stores the installvcs-universaluniqueidentifier response file inthe /op

Aaboutglobal clusters 17addingusers 71adding nodeto a one-node cluster 143attributesUseFence 101Ccablescross-over Ethernet 123clustercreating a single

Ffibre channel 23GGABdescription 15manual configuration 149port membership information 116verifying 116gabconfig command 116, 149-a (verifying GAB) 11

llttab fileverifying after installation 105MMAC addresses 40main.cf filecontents after installation 108MANPATH variablesetting 47manual installationpr

starting installationinstallvcs program 62Veritas product installer 62starting VCS 77storagefully shared vs. distributed 14shared 14switches 40Symante

About I/O fencingI/O fencing protects the data on shared disks when nodes in a cluster detect achange in the cluster membership that indicates a split

Figure 1-4Typical VCS setup with optional componentsSymantec ProductAuthentication Serviceroot brokerVCS cluster 1 VCS cluster 2VCS Management Console

Veritas Cluster Server Installation GuideThe software described in this book is furnished under a license agreement and may be usedonly in accordance

See “Preparing to configure the clusters in secure mode” on page 29.About Cluster Manager (Java Console)Cluster Manager (Java Console) offers complete

configurations for Windows, Linux, and Solaris clusters. VCS Simulator alsoenables creating and testing global clusters.You can administer VCS Simulat

Introducing Veritas Cluster ServerAbout VCS optional components22

Planning to install VCSThis chapter includes the following topics: About planning to install VCS Hardware requirements Supported operating systems

Table 2-1Hardware requirements for a VCS clusterDescriptionItemFrom 1 to 32 Linux PPC systems running the supported Linux PPCoperating system version.

Note: If you do not have enough free space in /var, then use the installvcscommand with tmppath option. Make sure that the specified tmppath file syst

Required Linux RPMs for VCSMake sure you installed the following operating system-specific RPMs on thesystems where you want to install or upgrade VCS

Supported softwareVeritas Cluster Server supports the previous and next versions of StorageFoundation to facilitate product upgrades, when available.V

Planning to install VCSSupported software28

Preparing to install VCSThis chapter includes the following topics: About preparing to install VCS Preparing to configure the clusters in secure mod

Symantec Corporation350 Ellis StreetMountain View, CA 94043http://www.symantec.com

The system clocks of the root broker and authentication brokers must be insync.The installvcs program provides the following configuration modes:The

Figure 3-1Workflow to configure VCS cluster in secure modeConfigure the cluster in secure modeReview AT concepts and gather required informationInstal

Table 3-1Preparatory tasks to configure a cluster in secure modeWho performsthis taskTasksVCS administratorDecide one of the following configuration m

Installing the root broker for the security infrastructureInstall the root broker only if you plan to use AT to configure the cluster in securemode. T

9Enter y when the installer prompts you to configure the Symantec ProductAuthentication Service.10Press the Enter key to start the Authentication Serv

If the output displays the following error, then the account for the givenauthentication broker is not created on this root broker:"Failed To G

The value for the authentication broker identity, whichyou provided to create authentication broker principalon the root broker system.This is the val

Note that for security purposes, the command to create the output file forthe encrypted file deletes the input file.5For each node in the cluster, cre

Do the following: Copy the root_hash file that you fetched to the system fromwhere you plan to install VCS.Note the path of the root hash file that y

Table 3-2Preinstallation tasks (continued)ReferenceTaskSee “Optimizing LLT media speed settings on private NICs”on page 48.Review basicinstructions to

Technical SupportSymantec Technical Support maintains support centers globally. TechnicalSupport’s primary role is to respond to specific queries abou

You can only install the Symantec software products for which you have purchaseda license. The enclosed software discs might include other products fo

Figure 3-3Private network setup with crossed linksCrossed linkPublic networkPrivate networksTo set up the private network1Install the required network

The systems can access the shared storage.4Test the network connections. Temporarily assign network addresses anduse telnet or ping to verify commun

To configure persistent interface names for network devices1Navigate to the hotplug file in the /etc/sysconfig directory:# cd /etc/sysconfig2Open the

collisions:0 txqueuelen:1000RX bytes:35401016 (33.7 Mb) TX bytes:999899 (976.4 Kb)Base address:0xdce0 Memory:fcf20000-fcf40000If a file named etc/sysc

where you run installvcs program. This privilege facilitates to issue ssh or rshcommands on all systems in the cluster. If ssh is used to communicate

4When the command prompts, enter a passphrase and confirm it.5Change the permissions of the .ssh directory by typing:# chmod 755 ~/.ssh6The file ~/.ss

See also the Veritas Cluster Server User's Guide for a description of I/O fencing.Setting the PATH variableInstallation commands as well as other

Setting the kernel.panic tunableBy default, the kernel.panic tunable is set to zero. Therefore the kernel does notreboot automatically if a node panic

Mounting the product discYou must have superuser (root) privileges to load the VCS software.To mount the product disc1Log in as superuser on a system

Version and patch level Network topology Router, gateway, and IP address information Problem description: Error messages and log files Troubles

To check the systems1Navigate to the folder that contains the installvcs program.See “Mounting the product disc” on page 49.2Start the pre-installatio

Installing and configuringVCSThis chapter includes the following topics: About installing and configuring VCS Getting your VCS installation and conf

Getting your VCS installation and configurationinformation readyThe VCS installation and configuration program prompts you for informationabout certai

To configure VCS clusters in secure mode (optional), you need: The name of the Root Broker systemExample: eastSee “About Symantec Product Authentic

The domain-based address ofthe SMTP serverThe SMTP server sends notification emails about theevents within the cluster.Example: smtp.symantecexample.c

VRTSvcsmn — Manual pages for VCS commandsAbout the VCS installation programYou can access the installvcs program from the command line or through th

Table 4-1installvcs optional features (continued)ReferenceOptional actionSee “Installing VCS with a response filewhere ssh or rsh are disabled” on pag

installvcs [ system1 system2... ] [ options ]Table 4-2 lists the installvcs command options.Table 4-2installvcs optionsDescriptionOption and SyntaxCon

Table 4-2installvcs options (continued)DescriptionOption and SyntaxSpecifies that the optional product RPMs such as man pagesand documentation need no

Table 4-2installvcs options (continued)DescriptionOption and SyntaxSpecifies that rsh and rcp are to be used for communicationbetween systems instead

Maintenance agreement resourcesIf you want to contact Symantec regarding an existing maintenance agreement,please contact the maintenance agreement ad

cluster configuration. The installvcs program prompts for cluster information,and creates VCS configuration files without performing installation.See

Table 4-3Installation and configuration tasksReferenceTask See “Starting the software installation” on page 61. See “Specifying systems for installa

Note: The system from where you install VCS must run the same Linux distributionas the target systems.To install VCS using the product installer1Confi

To specify system names for installation1Enter the names of the systems where you want to install VCS.Enter the system names separated by spaces on wh

3Enter keys for additional product features.Do you want to enter another license key for galaxy? [y,n,q,?](n) yEnter a VCS license key for galaxy: [?]

To install VCS RPMs1Review the output as the installer checks the RPMs that are already installed.2Choose the VCS RPMs that you want to install.Select

To install VCS packages now and configure VCS later1If you do not want to configure VCS now, enter n at the prompt.Are you ready to configure VCS? [y,

To configure VCS using the installvcs program1Confirm that you are logged in as the superuser and mounted the productdisc.2Navigate to the folder that

To configure the cluster1Review the configuration instructions that the installer presents.2Enter the unique cluster name and cluster ID.Enter the uni

5Choose whether to use the same NIC details to configure private heartbeatlinks on other systems.Are you using the same NICs for private heartbeat lin

Technical Support ... 4Chapter 1 Introducing Veritas Clust

Enter the name of the Root Broker system whenprompted.Requires a remote access to the Root Broker.Review the output as the installer verifies communic

See Veritas Cluster Server User's Guide for more information.Adding VCS usersIf you have enabled Symantec Product Authentication Service, you do

Refer to the Veritas Cluster Server User’s Guide for more information.To configure SMTP email notification1Review the required information to configur

If you do not want to add, answer n.Would you like to add another SMTP recipient? [y,n,q,b] (n)5Verify and confirm the SMTP notification information

Enter the SNMP console system name: [b,?] saturn Enter the minimum security level of messages to be sent to each console.Enter the minimum severity o

information to the VCS configuration file. You must perform additionalconfiguration tasks to set up a global cluster.See Veritas Cluster Server User&a

for installation are not met, the utility stops and indicates the actions that arerequired to proceed with the process. Review the output as the insta

CPI WARNING V-9-122-1021No PERSISTENT_NAME set for NIC with MAC address00:11:43:33:17:28 (present name eth0), though config file exists!CPI WARNING V-

Table 4-4File descriptionDescriptionFile Lists the RPMs that are installed on each system. Describes the cluster and its configured resources. Prov

Figure 4-2Client communication with LDAP serversVCS node(authentication broker)VCS client1. When a user runs HAcommands, AT initiates userauthenticati

Creating authentication broker accounts on root brokersystem ... 34Creating en

Distinguished name for the user container (for example,UserBaseDN=ou=people,dc=comp,dc=com) Distinguished name for the group container (for example

Note: Symantec recommends using Pentium III, 400MHz, 256MB RAM, and800x600 display resolution.The version of the Java™ 2 Runtime Environment (JRE) req

3Go to \windows\VCSWindowsInstallers\ClusterManager.4Open the language folder of your choice, for example EN.5Double-click setup.exe.6The Veritas Clus

To install VCS Simulator on Windows systems1Insert the VCS installation disc into a drive.2Navigate to the path of the Simulator installer file:\your_

Verifying the cluster after installationWhen you have used installvcs program and chosen to configure and start VCS,VCS and all components are properl

Reserved = 0Mode = VCSUpdating product licenses using vxlicinstYou can use the vxlicinst command to add the VCS license key on each node. Ifyou have V

4Make sure demo licenses are replaced on all cluster nodes before startingVCS.# cd /opt/VRTS/bin# ./vxlicrep5Start VCS on each node:# hastartAccessing

Configuring VCS clustersfor data integrityThis chapter includes the following topics: About configuring VCS clusters for data integrity About I/O fe

If a system is so busy that it appears to stop responding, the other nodes coulddeclare it as dead. This declaration may also occur for the nodes that

Disks that act as coordination points are called coordinator disks. Coordinatordisks are three standard disks or LUNs set aside for I/O fencing during

Adding VCS users ... 71Configuring SMTP email notification ...

Figure 5-1Workflow to configure disk-based I/O fencingInitialize disks as VxVM disksIdentify disks to use as coordinator disksSet up coordinator disk

For the latest information on supported hardware visit the following URL:http://entsupport.symantec.com/docs/283161 Each of the coordinator disks mus

When you run the vxfen startup file to start I/O fencing,the script creates this /etc/vxfentab file on each node witha list of all paths to each coord

Refer to the installation guide that comes with the Storage Foundation productthat you use.Perform the following preparatory tasks to configure I/O fe

3Verify that the ASL for the disk array is installed on each of the nodes. Runthe following command on each node and examine the output to verify thei

4Scan all disk drives and their attributes, update the VxVM device list, andreconfigure DMP with the new devices. Type:# vxdisk scandisksSee the Verit

command option verifies that the same serial number for the LUN is returned onall paths to the LUN.Make sure to test the disks that serve as coordinat

The same serial number information should appear when you enter theequivalent command on node B using the /dev/sdy path.On a disk from another manufac

If you use rsh for communication:# /opt/VRTSvcs/vxfen/bin/vxfentsthdw -n3The script warns that the tests overwrite data on the disks. After you revi

Setting up disk-based I/O fencing manuallyMake sure you completed the preparatory tasks before you set up I/O fencing.Tasks that are involved in setti