Posts Tagged ‘avid’

Xsan & Media Composer

Monday, April 22nd, 2013

There’s a product out there called SANFusion that can allow Media Composer to read disk images on Xsan as though each were a separate workspace in AvidFS. This allows environments to leverage existing pools of storage sitting on Xsan as though they were sitting on an Isis or a Terablock. Check it out at SANFusion:

SAN Fusion is an easy and cost-effective solution that allows you to use Avid Media Composer with your existing Xsan infrastructure.

By using an existing Xsan volume as a backing-store for SAN Fusion’s virtualized workspaces, editors can experience authentic Unity-style bin sharing and locking from within Media Composer 5.5.3 or later. Unlike other solutions that purport to make Avid work with Xsan, SAN Fusion is a client-side application with no additional server or minimum number of seats to buy.

SAN Fusion clients mount and unmount workspaces on demand through our intuitive GUI to provide Media Composer a real-time translation layer between it and Apple’s Xsan filesystem. The end result is the best of both worlds: Avid’s first class bin and media sharing combined with Apple’s storage-agnostic low (or no) cost cluster filesystem.

The $1,500 price tag shouldn’t scare you off. Just compare the cost of a Promise X30 to the same amount of storage from Avid and you’ll get the idea why. And if you need any help with such things, give us a shout at

Unity Best Practices In AVID Environments

Thursday, September 6th, 2012

Avid Unity environments are still common these days because the price for Avid’s ISIS SAN is tremendously high. While a Unity typically started anywhere from $50,000 to $100,000, a typical ISIS starts around the same price even though the ISIS is based on more typical, less expensive commodity hardware. The ISIS is based on common gigabit networking, whereas the Unity is based on fibre channel SCSI.

Avid Unity systems come in two flavors. Both can be accessed by fibre channel or by gigabit ethernet. The first flavor is all fibre channel hardware. The second uses a hardware RAID card in a server enclosure with a sixteen drive array and shares that storage over fibre channel and/or gigabit ethernet.

Components in a fibre channel only Unity can be broken down so:

  • Avid Unity clients
  • Fibre channel switch
  • Fibre channel storage
  • Avid Unity head

Components in a chassis-based Unity are:

  • Avid Unity clients
  • Fibre channel switch
  • Avid Unity controller with SATA RAID

The fibre channel only setup can be more easily upgraded. Because such setups are generally older, they typically came with a 2U rackmount dual Pentium 3 (yes, Pentium 3!) server. They use a 2 gigabit ATTO fibre channel card and reliability can be questionable after a decade.

The Unity head can be swapped for a no-frills Intel machine (AMD doesn’t work, and there’s not enough time in the world to figure out why), but one must take care to be careful about video drivers. Several different integrated video chips and several video cards have drivers which somehow conflict with Unity software, so sometimes it’s easier to simply not install any drivers since nothing depends on them. The other requirements / recommendations are a working parallel port (for the Unity dongle), a PCIe slot (for a 4 gigabit ATTO fibre channel card) and 4 gigs of memory (so that Avid File Manager can use a full 3 gigabytes).

The fibre channel switch is typically either a 2 gigabit Vixel switch or a 4 gigabit Qlogic 5200 or 5600 switch. The older Vixel switches have a tendency to fail because there are little heat sinks attached to each port chip which face downward, and after a while sometimes a heat sink or two fall off and the chip dies. Since Vixel is not in business, the only replacement is a Qlogic.

The fibre channel storage can be swapped for a SATA-fibre RAID chassis so long as the chassis supports chopping up RAID sets into many smaller logical drives on separate LUNs. Drives which Avid sells can be as large as 1 TB if using the latest Unity software, so dividing up the storage into LUNs no larger than 1 TB is a good idea.

Changing storage configuration while the Unity has data is typically not done due to the complexity and lack of proper understanding of what it entails. If it’s to be done, it’s typically safer to use a client or multiple clients to back up all the Unity workspaces to normal storage, then reconfigure the Unity’s storage from scratch. If that is what is done, that’s the best opportunity to add storage, change from fibre channel drives to RAID, take advantage of RAID-6, et cetera.

Next up is how Avid uses storage. The Unity essentially thinks that it’s given a bunch of drives. Drives cannot easily be added, so the only time to change total storage is when the Unity will be reconfigured from scratch.

The group of all available drives is called the Data Drive Set. There is only one Data Drive Set and it has a certain number of drives. You can create a Data Drive Set with different sized drives, but there needs to be a minimum of four drives of the same size to make an Allocation Group. Spares can be added so that detected disk failures can trigger a copy of a failing drive to a spare.

Once a Data Drive Set is created, the File Manager can be started and Allocation Groups can be created. The reasoning behind Allocation Groups is so that groups of drives can be kept together and certain workspaces can be put on certain Allocation Groups to maximize throughput and/or I/O.

There are pretty much two different families of file access patterns. One is pure video streaming which is, as one might guess, just a continuous stream of data with very little other file I/O. Sometimes caching parameters on fibre-SATA RAID are configured to have large video-only or video-primary drive sets (sets of logical volumes cut up from a single RAID set) are set to optimize streams. The other file access pattern would be handling lots of little files such as audio, stills, render files and project files. Caching parameters set for optimizing lots of small random file I/O can show a noticeable improvement, particularly for the Allocation Group which has the workspace on which the projects are kept.

Workspaces are what they sound like. When creating a workspace, you decide which Allocation Group that workspace will exist. Workspaces can be expanded and contracted even while clients are actively working in that workspace. The one workspace which matters most when it comes to performance is the projects workspace. Because Avid projects tend to have hundreds or thousands of little files, an overloaded Unity can end up taking tens of seconds to simply open a bin in Media Composer which will certainly affect editors trying to work. The Attic is kept on the projects workspace, too, unless explicitly set to a different destination.

Although Unity systems can have ridiculously long uptimes, like any filesystem there can be problems. Sometimes lock files won’t go away when they’re supposed to, sometimes there can be namespace collisions, and sometimes a Unity workspace can simply become slow without explanation. The simplest way to handle filesystem problems, especially since there are no filesystem repair tools, is to create a new workspace, copy everything out of the old workspace, then delete the old workspace. Fragmentation is not checkable in any way, so this is a good way to make a heavily used projects workspace which has been around for ages a bit faster, too.

Avids have always had issues when there are too many files in a single directory. Since the media scheme on Avids involves Media Composer creating media files in workspaces on its own, one should take care to make sure that there aren’t any single directories in media workspaces (heck, any workspaces) which have more than 5,000 files. Media directories are created based on the client computer’s name in the context of the Unity, so if a particular media folder has too many items, that folder can be renamed to the same name with a “-1″ at the end (or “-(n+1)”).

Avid has said that the latest Media Composer (6.0.3 at the time of this writing) is not compatible with the latest Unity client (5.5.3). This is not true and while certain exotic actions might not work well (uncompressed HD, large number of simultaneous multicam, perhaps), all basic editing functions work just fine.

Finally, it should be pointed out that when planning ways to back up Unity workspaces, Windows clients are bad candidates. Because of the limitation on the number of simultaneously mounted workspaces being dependent on the number of drive letters available, Windows clients can only back up at most 25 workspaces at a time. Macs have no limitation on the number of workspaces they can mount simultaneously, plus Macs have rsync built in to the OS, so they’re a more natural candidate for performing backups.

A Brief History of the Avid

Wednesday, October 12th, 2011

Avid started in the late1980s with proprietary hardware installed in a Macintosh II. In the early 1990s Avid systems used the NuVista card from TrueVision which was a motion JPEG card in Macintosh Quadra systems. These NuBus cards were then used in early PowerMacs, then made the transition to PCI cards. The systems were known as ABVBs (Avid Broadcast Video Board); NuBus ABVBs were often called Avid NuVista systems. PowerMacs with PCI ABVB cards are still in use today, although not commonly. They made the transition to HD because they allowed working in 23.98 FPS or 24 FPS, so onlining and conforming could be done by pretending the edit source was film and outputting cutlists accordingly.

The highest resolution supported by ABVB hardware is AVR-77, which was good enough for most people for broadcast SD (people had more modest standards in those days).

In 1998 Avid transitioned to the Meridien boardset. This new boardset was capable of handling uncompressed video, although Avid charged much more money for their Symphony model of the Meridien to enable uncompressed 23.98. The Symphony was sold as a finishing system, whereas the basic Meridien was sold as an offline editing system. Meridiens were also the first Avid boards which would work in x86 PC environments. In a move which upset many Avid users, Meridiens had no backwards compatibility with ABVB media and Avid stopped supporting ABVBs immediately after Meridiens were introduced. Considering that a typical ABVB setup cost around $40,000 new.

Avid introduced the Adrenaline products in 2003. This new software could run on a system which had no Avid hardware (and, of course, didn’t support any previous Avid hardware at all) or could be used with Avid “dongles”. Typically, a dongle is a plugin device used for copy protection and licensing, but Avid’s Mojo and Adrenaline audio/video interface boxes were hardly more than fancy Firewire video interfaces, so they were often deridingly referred to as Avid dongles. They provided no measurable advantage to an Avid beyond providing a means to connect a television monitor or decks for input and output.

In 2008, Avid introduced their Nitris DX and Mojo DX systems. Unlike the FireWire attached Adrenaline and Mojo “dongles”, these new DX systems were connected via PCIe and allowed for performing certain effects in realtime without the need for renders.

Throughout Avid’s history it reused many product names making it difficult to know with certainty to what a name is referring without context. For instance, the software with which the editor interacts has always been called Media Composer (although for a while there were two versions, Film Composer and Media Composer). These days an entire system might be referred to as a Media Composer system, but in the past it referred to just the software. To further complicate matters, Media Composer went through version numbers up through 5.6 on m68k Macs, through version 12 on PowerMacs, and then reset to version 1 with the introduction of the Adrenaline line of products.

The older families are clearly named. The earliest were referred to as NuVista, followed by ABVB, followed by Meridien. The Adrenaline family name is a bit confusing, since the name “Adrenaline” was used to refer to both the newer versions of Media Composer and the larger FireWire breakout box hardware. Mojo-based systems and software-only Media Composer systems of the same version (post version 12) were also called Adrenaline even when no Adrenaline hardware was present.

The newest systems use recycled names, too. The Meridien family had Symphony and Nitris versions which had additional features for color correction and finishing. Many still confuse Mojo DX and Mojo hardware which are completely different.

Early Avids used fast Avid labeled SCSI drives for storage of media. Meridiens allowed the use of internal IDE drives in PowerMacs, but this was discouraged by Avid because they had an interest in selling Avid drives. Media Composer was unable to work with FireWire drives as it caused Media Composer to crash. As time went on more people used RAID cards, SATA cards, and external arrays instead of Avid branded  SCSI drives.