C.1.1. Key Characteristics

This example configuration has the following key characteristics:

• Fencing device — An APC MasterSwitch (single-switch configuration). Refer to Table C-1 for switch information.

• Number of GFS nodes — 3. Refer to Table C-2 for node information.

• Number of lock server nodes — 3. The lock servers are run on the GFS nodes (embedded). Refer to Table C-2 for node information.

• Locking protocol — LOCK_GULM. The LOCK_GULM server is run on every node that mounts GFS.

• Number of shared storage devices — 2. Refer to Table C-3 for storage device information.

• Number of file systems — 2.

• File system names — gfs01 and gfs02.

• File system mounting — Each GFS node mounts the two file systems.

• Cluster name — alpha.

Notes

For shared storage devices to be visible to the nodes, it may be necessary to load an appropriate device driver. If the shared storage devices are not visible on each node, confirm that the device driver is loaded and that it loaded without errors. The small partition (/dev/sda1) is used to store the cluster configuration information. The two remaining partitions (/dev/sda2, sdb1) are used for the GFS file systems. You can display the storage device information at each node in your GFS cluster by running the following command: cat /proc/partitions. Depending on the hardware configuration of the GFS nodes, the names of the devices may be different on each node. If the output of the cat /proc/partitions command shows only entire disk devices (for example, /dev/sda instead of /dev/sda1), then the storage devices have not been partitioned. To partition a device, use the fdisk command.

C.1.2. Kernel Modules Loaded

Each node must have the following kernel modules loaded:

• gfs.o

• lock_harness.o

• lock_gulm.o

• pool.o

C.1.3. Setup Process

The setup process for this example consists of the following steps:

1. Create pool configurations for the two file systems.

   Create pool configuration files for each file system's pool: pool_gfs01 for the first file system, and pool_gfs02 for the second file system. The two files should look like the following:

   poolname pool_gfs01 subpools 1 subpool 0 0 1 pooldevice 0 0 /dev/sda2

   poolname pool_gfs02 subpools 1 subpool 0 0 1 pooldevice 0 0 /dev/sdb1

2. Create a pool configuration for the CCS data.

   Create a pool configuration file for the pool that will be used for CCS data. The pool does not need to be very large. The name of the pool will be alpha_cca. (The name of the cluster, alpha, followed by _cca). The file should look like the following:

   poolname alpha_cca subpools 1 subpool 0 0 1 pooldevice 0 0 /dev/sda1

3. Use the pool_tool command to create all the pools as follows:

   n01# pool_tool -c pool_gfs01.cf pool_gfs02.cf alpha_cca.cf Pool label written successfully from pool_gfs01.cf Pool label written successfully from pool_gfs02.cf Pool label written successfully from alpha_cca.cf

4. Activate the pools on all nodes.

   	Note
   	This step must be performed every time a node is rebooted. If it is not, the pool devices will not be accessible.

   Activate the pools using the pool_assemble -a command for each node as follows:

   n01# pool_assemble -a <-- Activate pools alpha_cca assembled pool_gfs01 assembled pool_gfs02 assembled n02# pool_assemble -a <-- Activate pools alpha_cca assembled pool_gfs01 assembled pool_gfs02 assembled n03# pool_assemble -a <-- Activate pools alpha_cca assembled pool_gfs01 assembled pool_gfs02 assembled

5. Create CCS files.

   1. Create a directory called /root/alpha on node n01 as follows:

      n01# mkdir /root/alpha n01# cd /root/alpha

   2. Create the cluster.ccs file. This file contains the name of the cluster and the name of the nodes where the LOCK_GULM server is run. The file should look like the following:

      cluster { name = "alpha" lock_gulm { servers = ["n01", "n02", "n03"] } }

   3. Create the nodes.ccs file. This file contains the name of each node, its IP address, and node-specific I/O fencing parameters. The file should look like the following:

      nodes { n01 { ip_interfaces { eth0 = "10.0.1.1" } fence { power { apc { port = 1 } } } } n02 { ip_interfaces { eth0 = "10.0.1.2" } fence { power { apc { port = 2 } } } } n03 { ip_interfaces { eth0 = "10.0.1.3" } fence { power { apc { port = 3 } } } } }

      Note

      If your cluster is running Red Hat GFS 6.0 for Red Hat Enterprise Linux 3 Update 5 and later, you can use the optional usedev parameter to explicitly specify an IP address rather than relying on an IP address from libresolv. For more information about the optional usedev parameter, refer to the file format in Figure 6-23 and the example in Example 6-26. Refer to Table 6-3 for syntax description of the usedev parameter.

   4. Create the fence.ccs file. This file contains information required for the fencing method(s) used by the GFS cluster. The file should look like the following:

      fence_devices { apc { agent = "fence_apc" ipaddr = "10.0.1.10" login = "apc" passwd = "apc" } }

6. Create the CCS Archive on the CCA Device.

   	Note
   	This step only needs to be done once and from a single node. It should not be performed every time the cluster is restarted.

   Use the ccs_tool command to create the archive from the CCS configuration files:

   n01# ccs_tool create /root/alpha /dev/pool/alpha_cca Initializing device for first time use... done.

7. Start the CCS daemon (ccsd) on all the nodes.

   	Note
   	This step must be performed each time the cluster is rebooted.

   The CCA device must be specified when starting ccsd.

   n01# ccsd -d /dev/pool/alpha_cca n02# ccsd -d /dev/pool/alpha_cca n03# ccsd -d /dev/pool/alpha_cca

8. At each node, start the LOCK_GULM server:

   n01# lock_gulmd n02# lock_gulmd n03# lock_gulmd

9. Create the GFS file systems.

   Create the first file system on pool_gfs01 and the second on pool_gfs02. The names of the two file systems are gfs01 and gfs02, respectively, as shown in the example:

   n01# gfs_mkfs -p lock_gulm -t alpha:gfs01 -j 3 /dev/pool/pool_gfs01 Device: /dev/pool/pool_gfs01 Blocksize: 4096 Filesystem Size:1963216 Journals: 3 Resource Groups:30 Locking Protocol:lock_gulm Lock Table: alpha:gfs01 Syncing... All Done n01# gfs_mkfs -p lock_gulm -t alpha:gfs02 -j 3 /dev/pool/pool_gfs02 Device: /dev/pool/pool_gfs02 Blocksize: 4096 Filesystem Size:1963416 Journals: 3 Resource Groups:30 Locking Protocol:lock_gulm Lock Table: alpha:gfs02 Syncing... All Done

10. Mount the GFS file systems on all the nodes.

    Mount points /gfs01 and /gfs02 are used on each node:

    n01# mount -t gfs /dev/pool/pool_gfs01 /gfs01 n01# mount -t gfs /dev/pool/pool_gfs02 /gfs02 n02# mount -t gfs /dev/pool/pool_gfs01 /gfs01 n02# mount -t gfs /dev/pool/pool_gfs02 /gfs02 n03# mount -t gfs /dev/pool/pool_gfs01 /gfs01 n03# mount -t gfs /dev/pool/pool_gfs02 /gfs02