Posts Tagged ‘qlogic’

Setting up a Qlogic Fibre Channel Switch For Xsan

Wednesday, April 11th, 2012

Qlogic switches can be configured via a built-in Web-based administration tool, or via their Command Line Interface over a serial connection. The Web-based tool is the fastest and easiest method of getting one up and running.

By default, Qlogic switches have an IP address of 10.0.0.1. The default username is “admin”, and the default password is “password”. Set your computer’s IP address to 10.0.0.2, with a Subnet Mask of 255.255.255.0 and no router/gateway. Open a web browser – Firefox is your best option – and go to 10.0.0.1. The Java applet will prompt a security warning – please confirm that the applet can control your computer. It won’t do anything bad.

On first logging in, you will be warned that the default password has not been changed. Please change the password. It’s very easy for somebody to make your fibre fabric not work right. Once you have done so, configure the IP address of the switch.

Please check and see if a firmware update is available for the switch before proceeding any further with setup. It’s definitely going to be easiesr to get a firmware update applied before you’ve got an Xsan using your fabric. Go to http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/NewDefault.aspx and click on Switches, then Fibre Channel Switches, choose the correct model, and click “Go”.

Devices on a fibre network are identified by their World Wide Name, or WWN. WWNs are guaranteed to be universally unique, which is a good thing, but they’re not designed to be read by humans. That’s why Qlogic lets you assign Nicknames to your devices. You should assign meaningful and easily decipherable Nicknames to all of your devices. Go to Fabric, and then Nicknames. You’ll see a list of all the WWNs (including vendor information), and which port they’re connected to. Double-click in the “Nickname” box, enter what you like, and when you’re done, click “Apply”. Accurate and comprehensible Nicknames make everything else easier, particularly the next step, which is Zoning.

Communication on a Fibre Channel network is controlled by Zones. In order for Fibre Channel devices to see one another (e.g. for clients to see storage), they must be in a zone together. In a small environment, it’s feasible to create a single zone, and place all devices in that zone. However, it isn’t necessary for Xsan clients and controllers to be able to communicate via Fibre Channel – all of their communication happens across the Metadata Network. If you want the best performance, then, it’s best to separate the devices logically into multiple zones to avoid excessive traffic on the Fibre Channel network. Devices can be added directly to a zone, or they can be grouped into Aliases, which can then be added to a zone.

As an example, imagine an environment with 15 Xsan clients, 2 Metadata controllers, and 2 Promise E-Class arrays. The clients need to communicate with the Promise storage, and the controllers do as well, but the clients and controllers don’t need to communicate with one another. Three aliases should be created and two zones should be created: one alias for each class of device, and one zone for each necessary communications channel.

Aliases

  • clients: Contains all Xsan clients
  • controllers: Contains both Metadata controllers
  • storage: Contains both Promises

Zones

  • XsanControllers: Contains the controllers and storage aliases
  • XsanClients: Contains the clients and storage aliases

Zones are contained in Zone Sets. Many Zone Sets can be configured, but only one Zone Set can be active at any time. Once you’ve created zones for your devices, put all those zones into a Zone Set, and make sure that you activate that Zone Set when you’re finished with your configuration changes.

Storage devices and clients on a Fibre Channel network present themselves to the switch differently, and require configuration specific to their role. There are port properties that need to be set to provide the best performance. Xsan controllers and clients are “Initiators”, and storage devices are “Targets”. Device Scan, when enabled, queries every newly connected device to determine whether or not it is a Target or an Initiator. I/O Streamguard attempts to prevent disruption by suppressing some types of communication between initiators. Since we know what every device will be, and what port they’re on, we can set Device Scan and I/O Streamguard appropriately and avoid the excess traffic.

Initiators: Enable I/O Streamguard Disable Device Scan Targets: Disable I/O Streamguard Enable Device Scan

Once you have your Nicknames, Zones, and port settings configured, you switch should be ready for use, and you can move on to configuring your storage, clients, and controllers.

Xsan Deployment Checklist

Tuesday, April 10th, 2012

One of the harder aspects of building systems consistently in a repeatable fashion is that you often need a checklist to follow in order to maintain that consistency. Therefore, we’ve started an Xsan Installation Checklist, which we hope will help keep all the i’s dotted and t’s crossed. Feel free to submit any items we should add to the checklist and also feel free to use it to verify the configuration of your own Xsans.

Preparation

[ ] Work out ahead of time how permissions will be dealt with:

  • Active Directory
  • Open Directory
  • Local Clients in same group with different UIDs.

[ ] If Active Directory is already in place, verify that system are bound properly.

[ ] If Open Directory is already in place, verify that system are bound properly.

[ ] If Open Directory is not already in place, configure Open Directory.

[ ] All client Public interfaces should have working forward and reverse DNS resolution.

Fibre Channel (Qlogic)

[ ] Update Qlogic firmware to latest on all switches.

[ ] Set nicknames for all devices in the fabric.

[ ] Export the nicknames.xml file and give to customer or import to workstation running Qlogic San Surfer.

[ ] Set the domain IDs on the Qlogic. Different Domain ID for each switch.

[ ] Set port speed manually on Qlogic and clients. Don’t use auto-negotiation.

[ ] Configure the appropriate Qlogic port properties for Targets (Storage) and Initiators (Clients).

Targets

  • Device Scan On
  • I/O Streamguard Off
  • Initiators
  • Device Scan Off
  • I/O Streamguard On

[ ] Avoid fully populating Qlogic 9200 blades, only use 8-12 ports of each blade to avoid flooding backplane.

[ ] If the switch has redundant power, plug each PS into different circuits.

[ ] Split HBA (client port) and storage ports across switches, i.e. port 0 on switch 1, port 1 on switch 2.

Storage (Promise)

[ ] Update Controller firmware to latest version

[ ] If client has a spare controller, update that as well.  Also label box with updated firmware number

[ ] Work out LUNs for MetaData/Journal and Data (MD should be RAID 1, Data should be RAID 5 or 6)

[ ] Adjust script for formatting Promise RAIDs Рrefer to this link  http://support.apple.com/kb/HT1200

[ ] Start formatting LUNS according to strategy – this can take up to 24 hours.

Metadata Network

[ ] If customer has Spanning Tree enabled, make sure Portfast is enabled as well. If possible, disable ST.

[ ] Verify that both clients and servers have GigE connection.

General Client/Server

[ ] Label your NICs clearly: Public LAN and Metadata LAN.

[ ] Configure Metadata network with IP and Subnet Mask only. No router or DNS.

[ ] Disable unused network interfaces.

[ ] Make sure Public Interface is top interface in System Preferences/Network

[ ] Disable IPv6 on all interfaces.

[ ] Energy Saver settings: Make sure “put hard disks to sleep when possible” is disabled.

[ ] Make sure Startup Disk is set to the proper local boot volume.

Metadata Controllers

[ ] Install XSAN on Snow Leopard machines and below (XSAN is included with Lion)

[ ] All MDCs should have mirrored boot drives, with AutoRebuild enabled.

[ ] Sync the clocks via NTP. Make sure all clients and MDCs point to same NTP server.

[ ] Add MDCs to XSAN

Volume Configuration

[ ] Label all the LUNs clearly.

[ ] Configure the MetaData LUN as a mirrored Raid 1.

[ ] Use an even number of LUNs per pool.

[ ] Use Apple defaults for block size and stripe breadth and test to see if performance is acceptable.

[ ] Do NOT enable Extended Attributes.

[ ] Verify email notification is turned on.

[ ] Make sure the customer knows not to go below 20% free space.

XSAN Creation/Management

[ ] Verify that the same version of Xsan is running on on all MDCs and clients.

[ ] For 10.6 and below – Add XSAN Serial numbers to XSAN Admin

[ ] Add Clients to XSAN

[ ] Verify performance of XSAN

  • Test speed
  • Test IO
  • Test sustained throughput
  • Test with different file types
  • Test within applications (real world testing)

[ ] Document XSAN for client

[ ] Upload documentation

 

Configuring a Qlogic Fibre Channel switch for Xsan

Tuesday, February 28th, 2012

Qlogic switches can be configured via a built-in Web-based administration tool, or via their Command Line Interface over a serial connection. The Web-based tool is the fastest and easiest method of getting one up and running.

By default, Qlogic switches have an IP address of 10.0.0.1. The default username is “admin”, and the default password is “password”. Set your computer’s IP address to 10.0.0.2, with a Subnet Mask of 255.255.255.0 and no router/gateway. Open a web browser – Firefox is your best option – and go to 10.0.0.1. The Java applet will prompt a security warning – please confirm that the applet can control your computer. It won’t do anything bad.

On first logging in, you will be warned that the default password has not been changed. Please change the password. It’s very easy for somebody to make your fibre fabric not work right. Once you have done so, configure the IP address of the switch.

Please check and see if a firmware update is available for the switch before proceeding any further with setup. It’s definitely going to be easiesr to get a firmware update applied before you’ve got an Xsan using your fabric. Go to Qlogic’s Support Site and click on Switches, then Fibre Channel Switches, choose the correct model, and click “Go”.

Devices on a fibre network are identified by their World Wide Name, or WWN. WWNs are guaranteed to be universally unique, which is a good thing, but they’re not designed to be read by humans. That’s why Qlogic lets you assign Nicknames to your devices. You should assign meaningful and easily decipherable Nicknames to all of your devices. Go to Fabric, and then Nicknames. You’ll see a list of all the WWNs (including vendor information), and which port they’re connected to. Double-click in the “Nickname” box, enter what you like, and when you’re done, click “Apply”. Accurate and comprehensible Nicknames make everything else easier, particularly the next step, which is Zoning.

Communication on a Fibre Channel network is controlled by Zones. In order for Fibre Channel devices to see one another (e.g. for clients to see storage), they must be in a zone together. In a small environment, it’s feasible to create a single zone, and place all devices in that zone. However, it isn’t necessary for Xsan clients and controllers to be able to communicate via Fibre Channel – all of their communication happens across the Metadata Network. If you want the best performance, then, it’s best to separate the devices logically into multiple zones to avoid excessive traffic on the Fibre Channel network. Devices can be added directly to a zone, or they can be grouped into Aliases, which can then be added to a zone.

As an example, imagine an environment with 15 Xsan clients, 2 Metadata controllers, and 2 Promise E-Class arrays. The clients need to communicate with the Promise storage, and the controllers do as well, but the clients and controllers don’t need to communicate with one another. Three aliases should be created and two zones should be created: one alias for each class of device, and one zone for each necessary communications channel.

  • Aliases
    1. clients: Contains all Xsan clients
    2. controllers: Contains both Metadata controllers.
    3. storage: Contains both Promises
  • Zones
    1. XsanControllers: Contains the controllers and storage aliases
    2. XsanClients: Contains the clients and storage aliases

Zones are contained in Zone Sets. Many Zone Sets can be configured, but only one Zone Set can be active at any time. Once you’ve created zones for your devices, put all those zones into a Zone Set, and make sure that you activate that Zone Set when you’re finished with your configuration changes.

Storage devices and clients on a Fibre Channel network present themselves to the switch differently, and require configuration specific to their role. There are port properties that need to be set to provide the best performance. Xsan controllers and clients are “Initiators”, and storage devices are “Targets”. Device Scan, when enabled, queries every newly connected device to determine whether or not it is a Target or an Initiator. I/O Streamguard attempts to prevent disruption by suppressing some types of communication between initiators. Since we know what every device will be, and what port they’re on, we can set Device Scan and I/O Streamguard appropriately and avoid the excess traffic.

  • Initiators:
    • Enable I/O Streamguard
    • Disable Device Scan
  • Targets:
    • Disable I/O Streamguard
    • Enable Device Scan

Once you have your Nicknames, Zones, and port settings configured, you switch should be ready for use, and you can move on to configuring your storage, clients, and controllers.