Posts Tagged ‘Xsan’

Files Not Showing For Xsan Clients When Uploaded Through Ethernet

Tuesday, October 15th, 2013

There is a problem with Xsan when using AFP or SMB heads in front of volumes, where when a user uploads or adds a file to the volume, the file is not readily available/visible to all users. This issue doesn’t occur every time a file is uploaded and nor does it cause files to actually disappear, only to need the user to restart their Finder in order to be able to see the object.

We’ve been using this freeware app as a workaround until Apple comes up with a patch: https://www.macupdate.com/app/mac/24714/refresh-finder

Xsan & Media Composer

Monday, April 22nd, 2013

There’s a product out there called SANFusion that can allow Media Composer to read disk images on Xsan as though each were a separate workspace in AvidFS. This allows environments to leverage existing pools of storage sitting on Xsan as though they were sitting on an Isis or a Terablock. Check it out at SANFusion:

SAN Fusion is an easy and cost-effective solution that allows you to use Avid Media Composer with your existing Xsan infrastructure.

By using an existing Xsan volume as a backing-store for SAN Fusion’s virtualized workspaces, editors can experience authentic Unity-style bin sharing and locking from within Media Composer 5.5.3 or later. Unlike other solutions that purport to make Avid work with Xsan, SAN Fusion is a client-side application with no additional server or minimum number of seats to buy.

SAN Fusion clients mount and unmount workspaces on demand through our intuitive GUI to provide Media Composer a real-time translation layer between it and Apple’s Xsan filesystem. The end result is the best of both worlds: Avid’s first class bin and media sharing combined with Apple’s storage-agnostic low (or no) cost cluster filesystem.

The $1,500 price tag shouldn’t scare you off. Just compare the cost of a Promise X30 to the same amount of storage from Avid and you’ll get the idea why. And if you need any help with such things, give us a shout at sales@318.com.

Setting up a Qlogic Fibre Channel Switch For Xsan

Wednesday, April 11th, 2012

Qlogic switches can be configured via a built-in Web-based administration tool, or via their Command Line Interface over a serial connection. The Web-based tool is the fastest and easiest method of getting one up and running.

By default, Qlogic switches have an IP address of 10.0.0.1. The default username is “admin”, and the default password is “password”. Set your computer’s IP address to 10.0.0.2, with a Subnet Mask of 255.255.255.0 and no router/gateway. Open a web browser – Firefox is your best option – and go to 10.0.0.1. The Java applet will prompt a security warning – please confirm that the applet can control your computer. It won’t do anything bad.

On first logging in, you will be warned that the default password has not been changed. Please change the password. It’s very easy for somebody to make your fibre fabric not work right. Once you have done so, configure the IP address of the switch.

Please check and see if a firmware update is available for the switch before proceeding any further with setup. It’s definitely going to be easiesr to get a firmware update applied before you’ve got an Xsan using your fabric. Go to http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/NewDefault.aspx and click on Switches, then Fibre Channel Switches, choose the correct model, and click “Go”.

Devices on a fibre network are identified by their World Wide Name, or WWN. WWNs are guaranteed to be universally unique, which is a good thing, but they’re not designed to be read by humans. That’s why Qlogic lets you assign Nicknames to your devices. You should assign meaningful and easily decipherable Nicknames to all of your devices. Go to Fabric, and then Nicknames. You’ll see a list of all the WWNs (including vendor information), and which port they’re connected to. Double-click in the “Nickname” box, enter what you like, and when you’re done, click “Apply”. Accurate and comprehensible Nicknames make everything else easier, particularly the next step, which is Zoning.

Communication on a Fibre Channel network is controlled by Zones. In order for Fibre Channel devices to see one another (e.g. for clients to see storage), they must be in a zone together. In a small environment, it’s feasible to create a single zone, and place all devices in that zone. However, it isn’t necessary for Xsan clients and controllers to be able to communicate via Fibre Channel – all of their communication happens across the Metadata Network. If you want the best performance, then, it’s best to separate the devices logically into multiple zones to avoid excessive traffic on the Fibre Channel network. Devices can be added directly to a zone, or they can be grouped into Aliases, which can then be added to a zone.

As an example, imagine an environment with 15 Xsan clients, 2 Metadata controllers, and 2 Promise E-Class arrays. The clients need to communicate with the Promise storage, and the controllers do as well, but the clients and controllers don’t need to communicate with one another. Three aliases should be created and two zones should be created: one alias for each class of device, and one zone for each necessary communications channel.

Aliases

  • clients: Contains all Xsan clients
  • controllers: Contains both Metadata controllers
  • storage: Contains both Promises

Zones

  • XsanControllers: Contains the controllers and storage aliases
  • XsanClients: Contains the clients and storage aliases

Zones are contained in Zone Sets. Many Zone Sets can be configured, but only one Zone Set can be active at any time. Once you’ve created zones for your devices, put all those zones into a Zone Set, and make sure that you activate that Zone Set when you’re finished with your configuration changes.

Storage devices and clients on a Fibre Channel network present themselves to the switch differently, and require configuration specific to their role. There are port properties that need to be set to provide the best performance. Xsan controllers and clients are “Initiators”, and storage devices are “Targets”. Device Scan, when enabled, queries every newly connected device to determine whether or not it is a Target or an Initiator. I/O Streamguard attempts to prevent disruption by suppressing some types of communication between initiators. Since we know what every device will be, and what port they’re on, we can set Device Scan and I/O Streamguard appropriately and avoid the excess traffic.

Initiators: Enable I/O Streamguard Disable Device Scan Targets: Disable I/O Streamguard Enable Device Scan

Once you have your Nicknames, Zones, and port settings configured, you switch should be ready for use, and you can move on to configuring your storage, clients, and controllers.

Xsan Deployment Checklist

Tuesday, April 10th, 2012

One of the harder aspects of building systems consistently in a repeatable fashion is that you often need a checklist to follow in order to maintain that consistency. Therefore, we’ve started an Xsan Installation Checklist, which we hope will help keep all the i’s dotted and t’s crossed. Feel free to submit any items we should add to the checklist and also feel free to use it to verify the configuration of your own Xsans.

Preparation

[ ] Work out ahead of time how permissions will be dealt with:

  • Active Directory
  • Open Directory
  • Local Clients in same group with different UIDs.

[ ] If Active Directory is already in place, verify that system are bound properly.

[ ] If Open Directory is already in place, verify that system are bound properly.

[ ] If Open Directory is not already in place, configure Open Directory.

[ ] All client Public interfaces should have working forward and reverse DNS resolution.

Fibre Channel (Qlogic)

[ ] Update Qlogic firmware to latest on all switches.

[ ] Set nicknames for all devices in the fabric.

[ ] Export the nicknames.xml file and give to customer or import to workstation running Qlogic San Surfer.

[ ] Set the domain IDs on the Qlogic. Different Domain ID for each switch.

[ ] Set port speed manually on Qlogic and clients. Don’t use auto-negotiation.

[ ] Configure the appropriate Qlogic port properties for Targets (Storage) and Initiators (Clients).

Targets

  • Device Scan On
  • I/O Streamguard Off
  • Initiators
  • Device Scan Off
  • I/O Streamguard On

[ ] Avoid fully populating Qlogic 9200 blades, only use 8-12 ports of each blade to avoid flooding backplane.

[ ] If the switch has redundant power, plug each PS into different circuits.

[ ] Split HBA (client port) and storage ports across switches, i.e. port 0 on switch 1, port 1 on switch 2.

Storage (Promise)

[ ] Update Controller firmware to latest version

[ ] If client has a spare controller, update that as well.  Also label box with updated firmware number

[ ] Work out LUNs for MetaData/Journal and Data (MD should be RAID 1, Data should be RAID 5 or 6)

[ ] Adjust script for formatting Promise RAIDs – refer to this link  http://support.apple.com/kb/HT1200

[ ] Start formatting LUNS according to strategy – this can take up to 24 hours.

Metadata Network

[ ] If customer has Spanning Tree enabled, make sure Portfast is enabled as well. If possible, disable ST.

[ ] Verify that both clients and servers have GigE connection.

General Client/Server

[ ] Label your NICs clearly: Public LAN and Metadata LAN.

[ ] Configure Metadata network with IP and Subnet Mask only. No router or DNS.

[ ] Disable unused network interfaces.

[ ] Make sure Public Interface is top interface in System Preferences/Network

[ ] Disable IPv6 on all interfaces.

[ ] Energy Saver settings: Make sure “put hard disks to sleep when possible” is disabled.

[ ] Make sure Startup Disk is set to the proper local boot volume.

Metadata Controllers

[ ] Install XSAN on Snow Leopard machines and below (XSAN is included with Lion)

[ ] All MDCs should have mirrored boot drives, with AutoRebuild enabled.

[ ] Sync the clocks via NTP. Make sure all clients and MDCs point to same NTP server.

[ ] Add MDCs to XSAN

Volume Configuration

[ ] Label all the LUNs clearly.

[ ] Configure the MetaData LUN as a mirrored Raid 1.

[ ] Use an even number of LUNs per pool.

[ ] Use Apple defaults for block size and stripe breadth and test to see if performance is acceptable.

[ ] Do NOT enable Extended Attributes.

[ ] Verify email notification is turned on.

[ ] Make sure the customer knows not to go below 20% free space.

XSAN Creation/Management

[ ] Verify that the same version of Xsan is running on on all MDCs and clients.

[ ] For 10.6 and below – Add XSAN Serial numbers to XSAN Admin

[ ] Add Clients to XSAN

[ ] Verify performance of XSAN

  • Test speed
  • Test IO
  • Test sustained throughput
  • Test with different file types
  • Test within applications (real world testing)

[ ] Document XSAN for client

[ ] Upload documentation

 

Configuring a Qlogic Fibre Channel switch for Xsan

Tuesday, February 28th, 2012

Qlogic switches can be configured via a built-in Web-based administration tool, or via their Command Line Interface over a serial connection. The Web-based tool is the fastest and easiest method of getting one up and running.

By default, Qlogic switches have an IP address of 10.0.0.1. The default username is “admin”, and the default password is “password”. Set your computer’s IP address to 10.0.0.2, with a Subnet Mask of 255.255.255.0 and no router/gateway. Open a web browser – Firefox is your best option – and go to 10.0.0.1. The Java applet will prompt a security warning – please confirm that the applet can control your computer. It won’t do anything bad.

On first logging in, you will be warned that the default password has not been changed. Please change the password. It’s very easy for somebody to make your fibre fabric not work right. Once you have done so, configure the IP address of the switch.

Please check and see if a firmware update is available for the switch before proceeding any further with setup. It’s definitely going to be easiesr to get a firmware update applied before you’ve got an Xsan using your fabric. Go to Qlogic’s Support Site and click on Switches, then Fibre Channel Switches, choose the correct model, and click “Go”.

Devices on a fibre network are identified by their World Wide Name, or WWN. WWNs are guaranteed to be universally unique, which is a good thing, but they’re not designed to be read by humans. That’s why Qlogic lets you assign Nicknames to your devices. You should assign meaningful and easily decipherable Nicknames to all of your devices. Go to Fabric, and then Nicknames. You’ll see a list of all the WWNs (including vendor information), and which port they’re connected to. Double-click in the “Nickname” box, enter what you like, and when you’re done, click “Apply”. Accurate and comprehensible Nicknames make everything else easier, particularly the next step, which is Zoning.

Communication on a Fibre Channel network is controlled by Zones. In order for Fibre Channel devices to see one another (e.g. for clients to see storage), they must be in a zone together. In a small environment, it’s feasible to create a single zone, and place all devices in that zone. However, it isn’t necessary for Xsan clients and controllers to be able to communicate via Fibre Channel – all of their communication happens across the Metadata Network. If you want the best performance, then, it’s best to separate the devices logically into multiple zones to avoid excessive traffic on the Fibre Channel network. Devices can be added directly to a zone, or they can be grouped into Aliases, which can then be added to a zone.

As an example, imagine an environment with 15 Xsan clients, 2 Metadata controllers, and 2 Promise E-Class arrays. The clients need to communicate with the Promise storage, and the controllers do as well, but the clients and controllers don’t need to communicate with one another. Three aliases should be created and two zones should be created: one alias for each class of device, and one zone for each necessary communications channel.

  • Aliases
    1. clients: Contains all Xsan clients
    2. controllers: Contains both Metadata controllers.
    3. storage: Contains both Promises
  • Zones
    1. XsanControllers: Contains the controllers and storage aliases
    2. XsanClients: Contains the clients and storage aliases

Zones are contained in Zone Sets. Many Zone Sets can be configured, but only one Zone Set can be active at any time. Once you’ve created zones for your devices, put all those zones into a Zone Set, and make sure that you activate that Zone Set when you’re finished with your configuration changes.

Storage devices and clients on a Fibre Channel network present themselves to the switch differently, and require configuration specific to their role. There are port properties that need to be set to provide the best performance. Xsan controllers and clients are “Initiators”, and storage devices are “Targets”. Device Scan, when enabled, queries every newly connected device to determine whether or not it is a Target or an Initiator. I/O Streamguard attempts to prevent disruption by suppressing some types of communication between initiators. Since we know what every device will be, and what port they’re on, we can set Device Scan and I/O Streamguard appropriately and avoid the excess traffic.

  • Initiators:
    • Enable I/O Streamguard
    • Disable Device Scan
  • Targets:
    • Disable I/O Streamguard
    • Enable Device Scan

Once you have your Nicknames, Zones, and port settings configured, you switch should be ready for use, and you can move on to configuring your storage, clients, and controllers.

The Impact of Directory Services on Xsan

Monday, January 23rd, 2012

When you’re dealing with file ownership and permissions, context is very important. Xsan volumes, from the point of view of an Xsan client, are local storage. There’s no daemon acting as gatekeeper or mediator, so when files are created or modified, the clients will use the standard mechanisms for assigning ownership and access rights, as they would with any local drive. In the absence of a shared authentication context, local user accounts will end up owning the files, and their permissions will be according to the default umask.

Mac OS X keeps track of such things by numerical User ID, and all Mac OS X systems start assigning local UIDs beginning with 501. It shouldn’be be too difficult to see how this can end badly. Users will potentially be able to overwrite files owned by other users, or access files they shouldn’t be able to, or not be able to access files that they should.

A directory service, therefore, is a key component in a proper deployment of Xsan. A directory service provides a shared context that will keep User ID collisions from happening. Also, users and groups can be managed centrally, and not on each workstation. This is especially important in large Xsan environments. Also, it will be possible to leverage Access Control Lists, and not POSIX permissions, to manage access to files and folders. POSIX permissions aren’t flexible enough to effectively manage the requirements of most Xsan environments.

It doesn’t matter whether you’re using Open Directory, Active Directory, or a Golden Triangle. You can even integrate Xsan with Novell’s eDirectory. Any of these will provide a much smoother and easy to manage Xsan.

Final Cut Pro X

Tuesday, September 20th, 2011

Version 10.0.1 of Final Cut Pro X is now out. This update returns the ability to use Final Cut Pro X projects and Events on Xsan. This is a must for multi-user environments. Users can now each others media and projects, and edit them from any system on the SAN, as with previous versions of Final Cut.

Additionally, some other new features including custom starting timecode, the new Tribute theme, GPU-accelerated exports, One-step transitions, media stems export and of course, XML support. XML support is very important as it introduces the ability to integrate Final Cut Pro X with asset management systems or APIs from other applications. The ability to interact with other tools helps to plan and implement an automated workflow, reducing the labor for reoccurring tasks common in media environments.


Apple also now provides a free 30 day trial to Final Cut Pro X. If your organization is considering migrating from Final Cut Studio into Final Cut Pro X, or if you have a Final Cut Server based asset management solution that you would like to migrate to something newer and supported, then please feel free to contact your 318 Professional Services Manager, or sales@318.com if you do not yet have one.

Video on Using Xsan in OS X Lion

Wednesday, July 20th, 2011

The All New Promise x30

Wednesday, April 13th, 2011

Yesterday, we mentioned Thunderbolt adapters for Xsan. But NAB 011 isn’t over and we have more announcements to bring up. Promise has announced their all new x30 series. With a spiffy new chassis design, these things now sport 8Gbps controllers, up to 48TB of space and up to 8 in a stack (that’s 7 expansion per chassis).

Oh, and we’d be remiss not to mention the redesigned management screen, a massive improvement over the command and control pane of glass we had before! A web-based management tool, by the way, that works on iPad! And management is easier now that you don’t have to restart the units every time you need to make a software update.

For more information on the new Promise x30, see:

http://www.promise.com/storage/raid_series.aspx?region=en-US&m=1053&sub_m=sub_m_8&rsn1=40&rsn3=48

To discuss how 318 can assist your organization in leveraging these new tools from Promise, from integrating a fleet of MacBook Pros with Xsan to bolting on additional storage for the always-full Xsan, contact your 318 Professional Services Manager, or sales@318.com if you do not yet have one!

Thunderbolt Adapters for Xsan

Tuesday, April 12th, 2011

As usual, there’s plenty to talk about after NAB. One of 2011′s favorites for us so far is the ability to attach a MacBook Pro to an Xsan using one of the newly introduced Thunderbolt -> Fibre Channel adapters.

The above adapter, from Promise, allows for 4Gbps and is “Fully Qualified” for Xsan. This allows the mobile user to be a first class citizen in a fibre channel SAN environment, not having to go through slow NAS heads to access large files, but instead connecting directly to Xsan or other fibre channel solutions!

For more, see http://www.promise.com/storage/raid_series.aspx?region=en-US&m=1054⊂_m=sub_m_8&rsn1=40&rsn3=49

Xsan 2.2.1

Friday, December 18th, 2009

Xsan 2.2.1 has been released. Updates include:

  1. Improved filesystem reliability
  2. Improved cvfsck (the Xsan filesystem repair tool)
  3. Resolves QuickTime reporting “invalid public movie atom found” on playback
  4. Eliminates “An unknown disk has been inserted” message when mounting Xsan volumes (occurs in Mac OS X 10.5 Leopard only)

Video: Creating a Device on Final Cut Server

Wednesday, July 29th, 2009

Final Cut Server on the Cheap

Monday, July 13th, 2009

At 318 we see a number of Final Cut Server installations. And for most of those jobs you should use an Xsan, have editors edit-in-place and develop custom automations. But Final Cut Server doesn’t have to be super complicated; nor does it have to be super expensive to integrate. At the end of the day it’s all about what a customer is expecting to get out of the product – and that’s how the product is developed and priced: to scale with the customers needs.

One of the most marketable and best features of Final Cut Server is that it is a way to catalog assets. These assets can be stored anywhere you want, provided that they are reliably accessible by the server. Given a username and password, users of Final Cut Server can then access the assets whether or not they can actually get to them using server shares or flat file systems. This allows Final Cut Server to bring logic to an otherwise chaotic form of storing data.

Once catalogued you can then tag assets with metadata. This means that when you go to find assets in the future you can do so quickly and easily. You can preview, annotate and then download those assets, no matter where they are stored – even on a Drobo or some large Firewire media sets. And if you decide to edit-in-place in the future, the fact that assets are stored in a logical space (called a Device) means that if you see the value, that you have an easy upgrade into more online media, such as an Xsan volume – but you don’t have to do it all at once to start seeing value.

And value is the key aspect of Final Cut Server. You can spend as much or as little as you need in order to get value out of the product. Sometimes the smallest features are what organizations will derive the most value out of. Not always, but sometimes… And when you see the value of the smaller features you can then make a decision based on your organizations goals and workflows what else will be of value. If you’d like to discuss a Final Cut Server implementation, whether it’s the basic initial installation, complicated workflow integration or custom scripting, contact 318 for more information. We’re here to help, whether or not the implementation is on the cheap.

Vmeter & Vguard for Xsan

Thursday, July 9th, 2009

Vmeter is another great product that you can bolt onto your Xsan from Vicom Systems. Vmeter allows you to get
Vmeter SQL Statistics statistics of bandwidth allocation for Xsan clients. But Vmeter doesn’t stop there. It also allows you to meter, or limit, the amount of bandwidth that is allocated to client machines, maximizing bandwidth for some users and tiering your performance allocation.

Vguard, also by Vicom Systems, is based on the technology included in Vmirror, the LUN mirroring solution, but goes a step further. Vguard allows you to setup another Xsan and use that SAN as a backup. We’re not going to go so far as to call it a snapshot, but it’s everything but.

Overall, Vicom integrates well with Xsan and fills some of the holes that the product itself has. For more information on Vmeter, Vguard or Vmirror, contact your 318 account manager today.

Xsan and Final Cut Server Monitors

Friday, May 1st, 2009

The Xsan and Final Cut Server monitors have been announced at Xsanity and are now available for download. These will monitor processor and memory utilization of the Xsan and Final Cut Server processes respectively. SSH tunneling will hopefully be added soon so that you can run them remotely but that’s closer to a 1.x release rather than the .x release that is available.

New article on Xsan Scripting by 318

Saturday, April 11th, 2009

318 has published another article on Xsanity, for scripting various notifications and monitors for Xsan and packaged up into a nice package installer. You can find it here
http://www.xsanity.com/article.php/20090407150134377.

Article on Xsanity – Linux + Xsan

Tuesday, January 13th, 2009

After a long silence on Xsanity, 318 has published the first of a number of articles for the site. The article focuses on how to install and configure StorNext clients running Red Hat Enterprise Linux (RHEL) to connect to an Xsan. It is available here.