6.7. Creating the nodes.ccs File

The nodes.ccs file specifies the nodes that run in a GFS cluster and their fencing methods. The nodes specified include those that run GFS and those that run LOCK_GULM servers. The nodes.ccs file is used in conjunction with the fence.ccs file to configure fencing in a cluster; the nodes.ccs file specifies fencing devices that are defined in the fence.ccs file.

Creating the nodes.ccs file consists of specifying the identity and fencing method (or methods) of each node in a GFS cluster. Specifying the identity consists of assigning a name and an IP address to the node. Specifying a fencing method consists of assigning a name to the fencing method and specifying its fencing-device parameters; that is, specifying how a node is fenced.

The way in which a fencing method is specified depends on if a node has either dual power supplies or multiple paths to FC storage. If a node has dual power supplies, then the fencing method for the node must specify at least two fencing devices — one fencing device for each power supply. Similarly, if a node has multiple paths to FC storage, then the fencing method for the node must specify one fencing device for each path to FC storage. For example, if a node has two paths to FC storage, the fencing method should specify two fencing devices — one for each path to FC storage. If a node has neither dual power supplies nor multiple paths to FC storage, then the fencing method for the node should specify only one fencing device.

You can configure a node with one fencing method or multiple fencing methods. When you configure a node for one fencing method, that is the only fencing method available for fencing that node. When you configure a node for multiple fencing methods, the fencing methods are cascaded from one fencing method to another according to the order of the fencing methods specified in the nodes.ccs file. If a node fails, it is fenced using the first fencing method specified in the nodes.ccs file for that node. If the first fencing method is not successful, the next fencing method specified for that node is used. If none of the fencing methods is successful, then fencing starts again with the first fencing method specified, and continues looping through the fencing methods in the order specified in nodes.ccs until the node has been fenced.

Refer to Chapter 10 Using the Fencing System for basic fencing details, descriptions of how fencing is used, and descriptions of available fencing methods.

To create the nodes.ccs file, follow these steps:

1. Create a file named nodes.ccs.

   1. If you are configuring a node for one fencing method (not cascaded), specify only one fencing method per node in the nodes.ccs file. Use a file format according to the fencing method as follows. Refer to Table 6-3 for syntax description.

      • APC MasterSwitch — For a node with a single power supply, refer to Figure 6-13. For a node with dual power supplies, refer to Figure 6-14.

      • WTI NPS — Refer to Figure 6-15.

      • Brocade, McData, or Vixel FC switch — Refer to Figure 6-16.

      • GNBD — Refer to Figure 6-17.

      • HP RILOE — Refer to Figure 6-18.

      • xCAT — Refer to Figure 6-19.

      • Egenera BladeFrame — Refer to Figure 6-20.

      • Manual — Refer to Figure 6-21.

        	Warning
        	Manual fencing should not be used in a production environment. Manual fencing depends on human intervention whenever a node needs recovery. Cluster operation is halted during the intervention.

   2. If you are configuring a node for cascaded fencing, use the file format in Figure 6-22. Refer to Table 6-3 for syntax description.

      	Note
      	Figure 6-22 does not show device-specific parameters for fencing methods. To determine device-specific parameters, refer to the appropriate figures listed in Step 1, part a.

2. For each node, specify NodeName, IFName, and the IP address of the node name, IPAddress.

   If your cluster is running Red Hat GFS 6.0 for Red Hat Enterprise Linux 3 Update 5 and later, you can use the optional usedev parameter to explicitly specify an IP address rather than relying on an IP address from libresolv. For more information about the optional usedev parameter, refer to the file format in Figure 6-23 and the example in Example 6-26. Refer to Table 6-3 for syntax description of the usedev parameter.

   	Note
   	Figure 6-23 and Example 6-26 do not show device-specific parameters for fencing methods. To determine device-specific parameters, refer to the appropriate figures listed in Step 1, part a.

   Note

   Make sure that you specify Nodename as the Linux hostname and that the primary IP address of the node is associated with the hostname. Specifying NodeName other than the Linux hostname (for example the interface name) can cause unpredictable results — especially if the node is connected to multiple networks. To determine the hostname of a node, use the uname -n command on the node. To verify the IP address associated with the hostname, issue a ping command to the hostname.

3. For each node, specify the fencing parameters according to the fencing method you are using, as follows:

   1. If using APC MasterSwitch fencing, specify MethodName, DeviceName, PortNumber, and SwitchNumber. If you are configuring for dual power supplies, specify the following parameters for the second fencing device: DeviceName, PortNumber, and SwitchNumber. Refer to Example 6-13 for a nodes.ccs file that specifies APC MasterSwitch fencing for a single power supply. Refer to Example 6-14 for a nodes.ccs file that specifies APC MasterSwitch fencing for dual power supplies.

   2. If using WTI NPS fencing, specify MethodName, DeviceName, and PortNumber. Refer to Example 6-15 for a nodes.ccs file that specifies WTI NPS fencing.

   3. If using Brocade, McData, or Vixel FC-switch fencing, specify MethodName, DeviceName, and PortNumber. If you are configuring for multiple paths to FC storage, specify the following parameters for each additional fencing device required: DeviceName and PortNumber. Refer to Example 6-16 for a nodes.ccs file that specifies Brocade FC-switch fencing. Refer to Example 6-17 for a nodes.ccs file that specifies McData FC-switch fencing. Refer to Example 6-18 for a nodes.ccs file that specifies Vixel FC-switch fencing.

   4. If using GNBD fencing, specify MethodName, DeviceName, and IPAddress. Refer to Example 6-19 for a nodes.ccs file that specifies GNBD fencing.

   5. If using HP RILOE fencing, specify MethodName, DeviceName, and PortNumber. Refer to Example 6-20 for a nodes.ccs file that specifies HP RILOE fencing.

   6. If using xCAT fencing, specify MethodName, DeviceName, and NodelistName. Refer to Example 6-21 for a nodes.ccs file that specifies xCAT fencing.

   7. If using Egenera BladeFrame fencing, specify MethodName, DeviceName, LPANName, and PserverName. Refer to Example 6-22 for a nodes.ccs file that specifies Egenera BladeFrame fencing.

   8. If using manual fencing, specify MethodName, DeviceName, and IPAddress. Refer to Example 6-23 for a nodes.ccs file that specifies manual fencing.

      	Warning
      	Manual fencing should not be used in a production environment. Manual fencing depends on human intervention whenever a node needs recovery. Cluster operation is halted during the intervention.

   9. If using cascaded fencing, specify parameters according to the type of fencing methods and in the order that the fencing methods are to cascade. Refer to Example 6-24 for a nodes.ccs file that specifies cascaded fencing.

   10. If using GNBD multipath, fence the GNBD server nodes using any of the fencing methods stated in previous steps in the procedure except for GNBD fencing (Step 3, part d). Refer to Example 6-25 for a nodes.ccs file that specifies fencing for a GNBD server node.

4. Save the nodes.ccs file.

	Warning
	Manual fencing should not be used in a production environment. Manual fencing depends on human intervention whenever a node needs recovery. Cluster operation is halted during the intervention.

nodes {
  NodeName {
     ip_interfaces {
         IFNAME="IPAddress"  <-- Must be an IP address; not a name
     }
     usedev = "NamedDevice"  <-- Optional parameter usedev
     fence {
     .
     .
     .
     }
  }
  NodeName {
  .
  .
  .
  }
}

nodes { 
   n01 { 
      ip_interfaces { 
         hsi0 = "10.0.0.1" 
      } 
      fence { 
         power { 
             apc1 { 
                port = 6 
                switch = 2 
             } 
          } 
       } 
  } 
  n02 {
  . 
  . 
  . 
  } 
}

nodes { 
   n01 { 
      ip_interfaces { 
        hsi0 = "10.0.0.1" 
      } 
      fence { 
         power { 
            apc1 { <----------- Fencing device for power supply 1
               port = 6 
               switch = 1 
               option = "off" <-- Power down power supply 1 
            } 
            apc2 { <----------- Fencing device for power supply 2 
               port = 7 
               switch = 2 
               option = "off" <-- Power down power supply 2
            } 
            apc1 { <----------- Fencing device for power supply 1
               port = 6 
               switch = 1 
               option = "on" <--- Power up power supply 1
            } 
            apc2 { <----------- Fencing device for power supply 2
               port = 7 
               switch = 2 
               option = "on" <--- Power up power supply 2
            } 
         } 
      } 
   } 
   n02 { 
   . 
   . 
   . 
   } 
}

nodes { 
   n01 { 
      ip_interfaces { 
         hsi0 = "10.0.0.1" 
      } 
      fence { 
         power { 
            wti1 { 
               port = 1 
            } 
         } 
      } 
   } 
   n02 { 
   . 
   . 
   . 
   } 
}

nodes { 
   n01 { 
      ip_interfaces { 
         hsi0 = "10.0.0.1" 
      } 
      fence { 
         san { 
            silkworm1 { 
               port = 3 
            } 
            silkworm2 { <--- Additional fencing device, for additional 
                port = 4     path to FC storage 
            } 
         } 
      } 
   } 
   n02 {
   . 
   . 
   . 
   } 
}

nodes { 
   n01 { 
      ip_interfaces { 
         hsi0 = "10.0.0.1" 
      } 
      fence { 
         san { 
            mdfc1 { 
               port = 3 
            } 
            mdfc2 { <--- Additional fencing device, for additional 
                port = 4     path to FC storage 
            } 
         } 
      } 
   } 
   n02 {
   . 
   . 
   . 
   } 
}

nodes { 
   n01 { 
      ip_interfaces { 
         hsi0 = "10.0.0.1" 
      } 
      fence { 
         san { 
            vixel1 { 
               port = 3 
            } 
            vixel2 { <---- Additional fencing device, for additional 
               port = 4    path to FC storage 
            } 
         } 
      } 
   } 
   n02 { 
   . 
   . 
   . 
   } 
}

nodes { 
   n01 { 
      ip_interfaces { 
         hsi0 = "10.0.0.1" 
      } 
      fence { 
         server { 
            gnbd { 
               ipaddr = "10.0.1.1" 
            } 
         } 
      } 
   } 
   n02 { 
   . 
   . 
   . 
   } 
}

nodes { 
   n01 { 
      ip_interfaces { 
         hsi0 = "10.0.0.1" 
      } 
      fence { 
         riloe { 
            riloe1 { 
               localport = 2345 
            } 
         } 
      } 
   } 
   n02 { 
   . 
   . 
   . 
   } 
}

nodes { 
   n01 { 
      ip_interfaces { 
         hsi0 = "10.0.0.1" 
      } 
      fence { 
         blade-center { 
                 xcat { 
                    nodename = "blade-01" 
            } 
         } 
      } 
   } 
   n02 {
      ip_interfaces { 
         hsi0 = "10.0.0.2" 
      } 
      fence { 
         blade-center { 
                 xcat { 
                    nodename = "blade-02" 
            } 
         } 
      } 
   } 
   n03 { 
   . 
   . 
   . 
   } 
}

nodes { 
   n01 { 
      ip_interfaces { 
         hsi0 = "10.0.0.1" 
      } 
      fence { 
         blade-center { 
                egenera { 
                   lpan = "opsgroup"
                   pserver = "ops-1
            } 
         } 
      } 
   } 
   n02 {
      ip_interfaces { 
         hsi0 = "10.0.0.2" 
      } 
      fence { 
         blade-center { 
                egenera { 
                   lpan = "opsgroup"
                   pserver = "ops-2
            } 
         } 
      } 
   }
   n03 { 
   . 
   . 
   . 
   } 
}

nodes { 
   n01 { 
      ip_interfaces { 
         hsi0 = "10.0.0.1" 
      } 
      fence { 
         human { 
            admin { 
               ipaddr = "10.0.0.1" 
            } 
         } 
      } 
   } 
   n02 { 
   . 
   . 
   . 
   } 
}	

	Warning
	Manual fencing should not be used in a production environment. Manual fencing depends on human intervention whenever a node needs recovery. Cluster operation is halted during the intervention.

   nodes { 
      n01 { 
         ip_interfaces { 
            eth0 = "10.0.1.21" 
         } 
         fence { 
            san {            <-- Fencing with Brocade FC switch
               brocade1 {      
                  port = 1 
               } 
            } 
            power {          <-- Fencing with APC MasterSwitch
               apc { 
                  port = 1 
                  switch = 1 
               } 
            } 
         }
      } 
      n02 { 
      . 
      . 
      . 
      } 
   }

nodes { 
   n01 { 
      ip_interfaces { 
         hsi0 = "10.0.0.1" 
      } 
      fence { 
         power { <------------- APC MasterSwitch fencing device
            apc1 { 
               port = 6 
               switch = 2 
            } 
      }
   } 
   n02 { 
   . 
   . 
   . 
   } 
}

nodes {
   n01 {
      ip_interfaces {
         wizzy = "10.0.0.1"  <-- Must be an IP address; not a name
      }
      usedev = "wizzy"  <-- Optional parameter usedev set to "wizzy"
      fence {
      .
      .
      .
      }
   }
   n02 {
   .
   .
   .
   }
}