Once you have read the Xsan Administration QuickStart and the Apple provided documentation and have achieved Xsan certification, then there are some fundamental items to understand about Xsan.
Chief amongst these are the benefits and deployments for Xsan.
Benefits to Xsan can be seen in one of three ways. The first is based on speed and the second on available hard drive space. The third, which is a more enterprise leaning approach is that the Xsan benefits can be obtained based on the virtualization of an organizations storage assets. In nearly every situation these will be the reasons that organizations choose to implement Xsan. Each of these will end up coming with its own set of business objectives and deployment methodologies.
If the SAN is being deployed simply for speed, such as the case where a post-production facility is seeking to perform High Definition video editing then it will often not matter the size of Xraids in deployment but instead more importance will be placed on the number of Xserve RAIDs and the amount of speed that can be aggregated amongst them.
When a SAN is deployed to provide space then the goal is to combine all of the data into one volume that is as large as possible. In an enviro nment where space is of the utmost concern and speed is not an issue, you can often have all of your LUNs the same large size and build metadata into them. The first LUN can be used for metadata and user data and then subsequent LUNs can be used for pure data. The space goal often coincides with the goal of virtualization.
Virtualization is the ability to “virtualize” or obtain otherwise unavailable flexibility from your storage. When virtualizing storage, administrators typically seek to have the ability to move LUNs from one location to another, add LUNs and reallocate storage to servers or workstations in a more dynamic manner. Here it is important to understand the limitations of the Xsan so as not to confuse the customer when you are discussing it with them. You can add storage to a volume without a reformat of the volume, appropriate space with the use of affinities that allocate storage to a specified host/server, appropriate speed to systems using storage pools and build quotas for all of the above into the system that will report when certain thresholds have been met.
Equally as important as knowing what you can do is knowing what you cannot or in some cases should not do. You cannot remove space of any kind from a volume without reformatting the volume. Additionally, you can’t swap two LUNs and have the data remain safe. In many SAN systems you can take a drive from anywhere in the SAN and plug it into another LUN and the SAN will remain unaffected. This is not possible in Xsan without breaking parity. Another limitation is with the addressing schemata of Fibre Channel as implemented in Xsan. You have a 64 node limitation. This can become frustrating considering that each WWNN is considered a node and many hosts will have 2. It is safe in many cases to think of a 32 device limitation rather than a 64 node limitation.
When you are working with Fibre Channel it is important to take the type of fabric that you are deploying into account. The Qlogic switches are 16 ports. If you have 8 devices with 2 active ports these will be full. The Qlogic switches are advertised as being 20 port switches. The reason for this is that they have 4 extra ports used for stacking switches exclusively. When you stack the switches you will be using a 10Gig uplink port to do so. Each switch can be stacked to 2 other switches in a fully fabric manner. In other words, the three switches act as one 48 port switch, supporting up to 24 devices (including he RAID and MDC devices in deployment. When stacking Fibre Channel devices it is typically referred to as cascading.
Distance is in nearly every Xsan deployment that we have done an isuse. When you purchase your Xserve RAID and Fibre Channel cards they will come with 7ft copper fibre cables. If you are going to place the devices up to 7 ft away from your Fibre Channel switch then you will need to adapt the copper-based fibre into glass-based fiber. When you do this you use a transceiver. Before purchasing any transceivers make sure they are supported by Apple. The default cable that comes with the Fibre Channel switches for cascading is also meant for short distances (less than 3 ft) and if you will be going further than this (such as across a street) then you will likely want to use transceivers to adapt that cable into 10gig fiber.
When a wiring engineer asks what kind of fiber you will be using, the typical answer is multi-mode sfp. If you are getting runs that provide for 10gig traffic then the answer would likely be more along the lines of multi-mode xfp. Mulit-mode only supports distances of 300 meters. If you will be going farther than that you will likely ask for single-mode fiber. It is important to note that 10Gig fiber is far more expensive to work with than 1Gig fiber. For example, a transceiver to adapt from 1Gig fiber to 1 Gig GBIC is around $75. However, the adapter to XFR would cost about $1,100. This is a large difference in cost. You also will need to be extra careful with the 10 Gig fiber as it is much more susceptible to breakage.
Three18 does not cut fiber (or copper cables). If a client asks if we can then we will refer them to one of our wiring partners. Do not look into a cable to see if you can see the light. Do not bend the cables in such a way that can break the shielding. Do not leave cable in a place it can be stepped on and do not pull it very hard when trying to get it to come loose from a port or a transceiver. With fibre you do not have to worry about ESD or EMI interference with the cable.
When importing data into the SAN it is faster to use the cvcp command rather than perform a drag and drop. Using the cvcp command it is also possible to maintain any attributes that would otherwise be lost during the transfer (eg- on Xsan previous to 1.3 this would include file date and time stamps).
Identity is also an issue with Xsan. Before you perform your first deployment you should already be familiar with setting up an Open Directory Master or joining the SAN systems into a pre-existing OD/AD environment. The ability for the SAN to identify the users is almost exclusively used for permissions. In no way will this affect the performance, speed or features of the SAN aside from actual permissions. However, the larger the SAN the more important the permissions will become. Using Open Directory it is possible to set the default group of all the SAN users to be the same, assign them a umask which allows editing the default permissions of files they add to the SAN and the default permissions levels.
Many clients who seek virtualization or space are often looking to use the SAN volume as space for a collection of servers. For example, if you have 5,000 users attempting to mount the same AFP volume then you are likely to have very poor performance. However, you could easily use a Coyote load balancer, round robin DNS, a BigIP or an F5 to direct traffic to a collection of servers that use the same SAN as their back-end storage. When we are working in these types of situation we typically refer to these servers as bridgehead servers. We would also use this term to describe any server resharing a service. In some cases, you should expect a degree of latency degradation when doing this.
One reason for this is that the actual file system used on a SAN volume is the ADIC SAN file system. For this reason, the standard tools used on a disk should not be run against a SAN. Fsck, Disk Warrior, the repair function of Disk Utility and Drive 10 should never be run on an Xsan volume. For Xsan volumes there is cvfsck and cvdefrag. Before running either of these utilities you should always backup.
There are many Xsan options available through the GUI but there are more available through the command line. In the beginning you will likely primarily only use cvfsck and cvcp but will later begin to learn the layout and usage of the configuration files. The command line reference for Xsan is a good place to get a strong understanding of what is and is not available through the command line.
It is possible to log other operating systems into Xsan using ADIC software. The client licenses are $1,750 per node. You still control the machine from within the Xsan Admin utility from a Metadata controller.