• Secure, Searchable Rushes/Dailies Ingest and Protection (Panasonic P2 etc)
• Nearline or parking space for AVID or Final Cut workflows
• Local Library, keeping assets alive to be re-purposed/re-used

The View from OM

NAB is always an experience and shows how niche storage vendors can dominate in a space not yet penetrated by the big tin companies (thought they all claim to be there). Please do get in touch if you would like to have a chat with Mark or myself.

The video (HD version available here) provides a high level view of The MatrixStore in this role:

As shown in the video there are 4 main benefit areas:

• Guaranteed Ingest - MatrixStore generates a hash (digest, digital fingerprint) of the content on the client and also on the two MatrixStore nodes the content lands on. Three different places where the content is checked ensuring the content in MatrixStore is bit for bit the same as came off the P2 or SxS card. Because MatrixStore keeps that digest/fingerprint as metadata with the content we are able to automatically, periodically and systematically (insert line from Grease here) ensure the content on disk matches the digest kept with it. Forever.. and ever.

• Secure Protection - MatrixStore provides the option to keep content in vaults that can have retention periods set upon them. No deletion or modification before the retention date. End of. This helps not only for the long term retention of content but also as a highly secure ingest vault for rushes/dailies coming into the post process. Set a one month retention policy on an ingest vault and that content will be secure from the accidents that do tend to happen.

• Searchable Metadata – “If you can’t find it you don’t have it”. DropSpot harvests metadata contained within the P2 (and other formats) media and creates a searchable rushes (dailies) repository allowing producers or editors to find and retrieve clips in an instant.

• Making use of the content – Now that the data is online and protected at the highest levels you can now access the data. Using either DropSpot or MxFS from any authenticated client content can be copied to and from the archive or can be accessed from within applications like Media Composer or Final Cut Pro. Using MxFS, The MatrixStore is perfect to provide quick access to content to browse or perform a quick edit before working ‘online’.

The View from OM

There is currently no de-facto standard or best practice defined for the safe ingest of tapeless content. The MatrixStore offers the ability to provide guaranteed ingest coupled with an audit trail from ingest through to archive in AVID and Final Cut Workflows.

During BVE last week I spent time chatting with Larry Jordan for the ‘The Digital Production Buzz’ podcast talking about:

• The two maxims .. now three…
• Highly secure protection of file-based rushes/dailies
• MatrixStore as nearline or parking space
• Keeping legacy assets online in order to re-use or re-purpose

We also touched on the integrations with CatDV/Final Cut Server and disk based archiving in general.

Click on the image to to to the website:

Larry  is an internationally-renowned consultant and Apple-Certified trainer in digital media with over 25 years experience as a television producer, director and editor with national broadcast and corporate credits. Based on Los Angeles, he’s a member of both the Directors Guild of America and the Producers Guild of America. Jordan is the Founder and Editor-in-Chief of Edit Well, the rich-media newsletter for Final Cut Studio, published by Peachpit Press. He is the author of hundreds of hours of online training, published by Lynda.com and two books on Final Cut Pro, published by Peachpit Press. Visit his website at www.larryjordan.biz.

The View From OM

I would like to thank Larry for once again giving us the opportunity to talk about MatrixStore and it position in video workflow. I will post about other nice stuff at BVE in another post.

The OM  team will demonstrate how The MatrixStore can protect content from ingest to archive:

- Guaranteed ingest with metadata harvesting (Panasonic P2)

- Secure Ingest Vaults (Lock down from accidental deletion)

- Production/Project parking to a Nearline Vault in AVID and Final Cut Workflows (one click operation)

- Archiving Legacy assets to an online Library Vault using DropSpot, Final Cut Server and CatDV

Implementing a tapeless or file based workflow has been highlighted as one of the top challenges facing creative organisations in 2010. With no tape to put on the shelf broadcasters, post-production facilities and creative agencies are looking for a solution to keep their assets secure and more importantly available throughout the production process. A single MatrixStore cluster acts as pre-production, nearline or local archive allowing content to move seamlessly through any digital workflow from ingest to delivery.

The View From OM

We are very much looking forward to the show so come along and see us on stand C10!

One sweet spot for MatrixStore technology is in the creative space. Creative organisations are increasingly being subjected to change driven by external forces such as changes in the way content is captured and our insatiable appetite for HD or 3D content (see The Big Broadcast Survey published in 2009). The industry is one that is used to adapting to whatever is thrown at it but it also relies on solutions that just work.

Below is a quick summary of the challenges they currently face and how MatrixStore is positioned to help. I will elaborate on each point in later posts :

1. File Based/Tapeless Ingest - Some call it pre-production archiving, some call it prechiving, the problem is the same, more and more content is being captured on disk or flash. There is no tape to put on the shelf. Protecting content that enters the building in an audited manner is a key area where MatrixStore is helping. Integration into Final Cut and Avid workflows, digital fingerprints on ingest, locked vaults to protect from deletion and metadata harvesting from P2, SxS etc are some of the benefits.

2. Parking/Nearline – Production disk, the online, the SAN should be kept lean and mean. Some projects can take years to complete so content needs to be moved off and onto the expensive disk repeatedly over a prolonged period. Tape based technologies do not cut the mustard. Organisations are also looking to ‘rightsize’ their expensive disk by offloading older or less-used content that should not sit clogging up the SAN to a more appropriate tier of storage. Seamless one touch capacity expansion, DropSpot (metadata support/search) and performance make MatrixStore a natural choice.

3. Legacy/Local Archive – organisations who own content need to protect their content or they will not be able to re-use or monetise that content. The two maxims come into their own here. Integration with DAM/MAMs, Self-Management,  redundancy of data, retention policies (prevent change or deletion), metadata and search goodness are key here.

4. Content Manipulation/Distribution Repository - If you own the content and have it protected and available on disk then what can you do with that data? Clustered/Grid architecture, high availability, metadata and search again key.

5. Regulations - More and more financial institutions are using video in the way they communicate with staff or shareholders, that video needs to be protected and stored for a pre-determined period in accordance with global regulations. MatrixStore has built in characteristics that satisfy these compliance requirements.

The View From OM

Even medium to large creative organisations will not have dedicated I.T staff to work on the video side of the business so they need solutions that just work and do not take them out of their comfort zone. The MatrixStore is the storage geek so they don’t need one.

Cardiff, UK – Object Matrix, a developer of secure asset archive and retrieval software, has
today supplied Cardiff based post-production facility Mwnci with its pre-production rushes archive and nearline archive solution, The MatrixStore. The multi-purpose data management
solution provides a secure archive for client owned content that enters the facility and also acts as a secure parking location for ongoing projects.

Rich Moss, Managing Director of Mwnci stated, “More and more of the projects coming into Mwnci are file based and as such we need to ensure we have the best infrastructure in place to cope with not only the incredible amount of content but also the wide variety of the content formats that come into the facility. The DropSpot software from Object Matrix understands the Panasonic P2 and Sony SxS formats allowing metadata entered on the camera to be harvested and attached to the content in our MatrixStore cluster. We made a decision to keep our production SAN at an optimal size for both performance and throughput whilst moving older content that still needs to be accessed to a more appropriate platform.

The MatrixStore fits perfectly as both the pre-production and the nearline archive in our Avid based workflow.”

Mwnci uses mostly Avid technology in the post production of television programming for clients such as the BBC and S4C.

More here….

The View From OM:
For more information on the use of MatrixStore in an AVID workflow go here..

Happy Days. A great end to 2009. Merry Christmas and wishing you a prosperous new year!

Adding to our proven Apple Final Cut Server and CatDV workflows we’ve made some recent improvements to using MatrixStore as both a Pre-Production Archive (ingest location) and a Nearline Archive in Avid workflows.

The workflow shown in this blog post uses Avid AMA and Avid Unity together with MatrixStore at a post-production house here in the UK.

The Phase 1 setup below was fully tested during whilst the items in phase 2 were left to be tested at a later stage (although they are assumed to be working and will be tested as required).

Phase 1 Usage

• Footage and metadata were ingested from P2 cards and local disks
• Windows clients (Vista 64 bit, XP 32-bit)
  
• 3-node MatrixStore Nearline cluster accessed from clients on a Gigabit network
• Avid AMA clients on Windows Vista 64-bit machines
• DropSpot (MatrixStore Client data browser application on Windows and Mac)
• Avid Unity on a FC network

Phase 2 (not tested but to be tested when required)

• Sony SxS cards
• Avid AMA Mac clients
• Panasonic P2 Viewer updating to MatrixStore
• Avid AMA building sequences to MatrixStore
• LTO tape writers running off MatrixStore via MXFS

The following diagram shows the above set-up:

Commentary and Results

The MatrixStore cluster and clients were installed, physically connected to the network and were usable within 3 hours of arriving onsite, with the main delay being working out which clients to use for which stages of the workflow.

Ingest media from P2 or disk was imported via the DropSpot application using the following steps:

1. Data folders to move the footage to were created in DropSpot.
2. Data was imported via DropSpot to the MatrixStore.
3. Multiple clients could Ingest in parallel.

During the Ingest of data, the metadata from the P2 cards was examined and added to the footage itself. This allows the footage to be searched for and also means that the metadata will be kept securely with the footage for as long the footage is kept in the MatrixStore Pre-Production/Nearline archive.

DropSpot runs on Mac, Windows and Linux, and MatrixStore was also found to be useful as a shared storage location between clients.

The resulting data movement occurred as fast as the data could be read. In the case where P2 cards were used this was approximately 50MB/s to 60MB/s, and in a test where data was read from a fast direct attached storage device, data moved at the full network card interface speed available.  Sony SxS cards are expected to ingest at a slower rate where they are using USB technology.

For Edition, Avid AMA clients were set-up on Windows Vista 64bit and XP 32 bit machines and used the MatrixStore “MXFS” virtual filesystem to view and update ingested data as if it were on an attached SAN device. This was achieved in two stages:

1. The folders of Ingested data were scanned by selecting the Avid AMA import volume option. Scanning a folder takes around 20-30 seconds per P2 card whilst Avid software browsed and created stubs.
2. Avid AMA was then to consolidate the data onto the Unity platform. Where consolidating didn’t transcode the data moved from the MatrixStore to Unity at full speed with the client CPU utilized seen to be approximately 10%. Where consolidation was involved transcoding the data moved at 25-30MB/s and the 8 cores of CPU were at 90%+ (ie the bandwidth was limited by the transcoding CPU speed).

Multiple clients can consolidate/send data to Unity in parallel.

One typical usage scenario of the MatrixStore Nearline used was that prior to ingest of data, the user would open up one to three data streams in an Avid AMA client to view the content. So long as the client could cope with the streams (e.g., the graphics card could cope) this worked very well.

Conclusion

• The set-up of the Nearline archive and clients was straightforward and resulted in the Nearline archive being ready to be used on the day that it arrived, validating the Object Matrix “plug and play” analogy.
• The main bottle-necks in the workflow were the speed of the cards being read from and the time taken to transcode data (the users were very happy to see their dual quad core clients being used to their capacity!).
• The new MatrixStore MXFS connection from Windows and Mac was straightforward to set-up and provided up to full bandwidth across the network.
• MatrixStore acted as a first class Nearline archive keeping data safe, secure and available throughout.

Future plans for Object Matrix include forthcoming direct support for Avid Interplay and further support for metadata extraction and handling, but if you’d like to know more about our reference sites and plans for future workflow support let us know!