- Preserving and managing rushes and work-in-progress in tapeless environments

- Tracking and exploiting metadata through workflows, including multi-site, multi-vendor workflows

- Cost effective production and media asset management in multi-vendor (Avid, Apple, Adobe, Autodesk to name just the “A”s) environments

- Multi-platform production/project management

Click below to see a time lapse video from the tech room:

The View from OM

This event was the first of many and the feedback was very encouraging in that our community consisting of technologists and creative professionals is ready and willing to collaborate to produce solutions that work.

Thanks to the Mediasmiths (Steve in particular) for running the event and to the other tech companies who took time out to share their thoughts and skills. (Gridiron Software, Vidispine, MXFServer, First Person and the BBC Ingex team)

(Music in video: Chepstow Male Voice Choir)

• Object Matrix showing The MatrixStore archive in a number of workflows (Final Cut Server, Avid, Pre-Production, Nearline, Backup etc)
• Vidispine showing its new Media Asset Management platform
• Gridrion Software showing Flow
• ERA will be demonstrating MXFServer from Film Partners.

The View from OM

We are looking forward to getting together with the likes of Simon, Eric, Isak, Sean and the Mediasmiths. The technologies are totally complimentary and offer the ability to automate end-to-end workflows with repeatable results. The blog from Steve is here and the invite to the event is here.

TCO Analysis Document (TCOAnalysis)

An overview of the elements one needs to consider when putting together a TCO model. The elements in this document were garnered with the help from the wider storage community and creative professionals. The content under those elements is our work and we hope to have avoided FUDlike comment.

TCO Spreadsheet (Disk Vs Tape TCO v2.1.1 2009 (version 1))

Open to scrutiny and allows the user to enter their own values. We have only put in one tape library that has been spec’d to grow at 50TB p.a as, surprise surprise, we have been unable to get accurate pricing on a solution like the Scalar i2000 which can scale to 4500 tape slots (just under 5PB). If anyone out there has that pricing and can plug it in please do so and report back here in the comments.

In the spreadsheet we have place holders for showing the benefit of having the data available online and available. When you start to input the manual costs of retrieving content the TCO swings further towards clustered disk.

The View From OM

Please take these docs, scrutinise them, love them, hate them, change them, share them, add products and pricing. It is one way as a community we can create a living TCO model.

Being a vendor of disk based solutions we are of course confident the numbers will stay in favour of disk. We more than happy to have a conversation around this subject and potentially walk through the documents in order to ensure you get the best out of them for your needs. Get in touch!

Previous Posts on TCO:

http://www.matrixstore.net/2009/09/17/defining-an-up-to-date-tco-model/

http://www.matrixstore.net/2008/03/03/problems-with-total-cost-of-ownership/

The OM  team will demonstrate how The MatrixStore can protect content from ingest to archive:

- Guaranteed ingest with metadata harvesting (Panasonic P2)

- Secure Ingest Vaults (Lock down from accidental deletion)

- Production/Project parking to a Nearline Vault in AVID and Final Cut Workflows (one click operation)

- Archiving Legacy assets to an online Library Vault using DropSpot, Final Cut Server and CatDV

Implementing a tapeless or file based workflow has been highlighted as one of the top challenges facing creative organisations in 2010. With no tape to put on the shelf broadcasters, post-production facilities and creative agencies are looking for a solution to keep their assets secure and more importantly available throughout the production process. A single MatrixStore cluster acts as pre-production, nearline or local archive allowing content to move seamlessly through any digital workflow from ingest to delivery.

The View From OM

We are very much looking forward to the show so come along and see us on stand C10!

One sweet spot for MatrixStore technology is in the creative space. Creative organisations are increasingly being subjected to change driven by external forces such as changes in the way content is captured and our insatiable appetite for HD or 3D content (see The Big Broadcast Survey published in 2009). The industry is one that is used to adapting to whatever is thrown at it but it also relies on solutions that just work.

Below is a quick summary of the challenges they currently face and how MatrixStore is positioned to help. I will elaborate on each point in later posts :

1. File Based/Tapeless Ingest - Some call it pre-production archiving, some call it prechiving, the problem is the same, more and more content is being captured on disk or flash. There is no tape to put on the shelf. Protecting content that enters the building in an audited manner is a key area where MatrixStore is helping. Integration into Final Cut and Avid workflows, digital fingerprints on ingest, locked vaults to protect from deletion and metadata harvesting from P2, SxS etc are some of the benefits.

2. Parking/Nearline – Production disk, the online, the SAN should be kept lean and mean. Some projects can take years to complete so content needs to be moved off and onto the expensive disk repeatedly over a prolonged period. Tape based technologies do not cut the mustard. Organisations are also looking to ‘rightsize’ their expensive disk by offloading older or less-used content that should not sit clogging up the SAN to a more appropriate tier of storage. Seamless one touch capacity expansion, DropSpot (metadata support/search) and performance make MatrixStore a natural choice.

3. Legacy/Local Archive – organisations who own content need to protect their content or they will not be able to re-use or monetise that content. The two maxims come into their own here. Integration with DAM/MAMs, Self-Management,  redundancy of data, retention policies (prevent change or deletion), metadata and search goodness are key here.

4. Content Manipulation/Distribution Repository - If you own the content and have it protected and available on disk then what can you do with that data? Clustered/Grid architecture, high availability, metadata and search again key.

5. Regulations - More and more financial institutions are using video in the way they communicate with staff or shareholders, that video needs to be protected and stored for a pre-determined period in accordance with global regulations. MatrixStore has built in characteristics that satisfy these compliance requirements.

The View From OM

Even medium to large creative organisations will not have dedicated I.T staff to work on the video side of the business so they need solutions that just work and do not take them out of their comfort zone. The MatrixStore is the storage geek so they don’t need one.

• File based/Tapeless workflows (P2, Sony, JVC)
• Automated Digital Asset Management (Final Cut Server, CatDV etc)
• Digital Preservation
• Disk based archiving
• Pre & post production archiving
• Rugby (Nick)
• Fooseball (Jon)

So pop over and say hello : Video Rescue Stand (6.C28d)

Cheers

N

One aspect of the role is very simple (DIT role is more complex, thanks JET and Filmbot):

- Take the content off the storage media (Red, P2, Sony)

- Copy the content to two locations (usually firewire drives)

- Erase the storage media 

- Give it back to be re-used

- Repeat

The problem with this approach is that is involves a lot of manual steps to protect the data, there is a real risk of content going missing (mislaid or worse) and also the volumes of data produced can create a management headache. 

For this reason Object Matrix have commissioned a 12Terabyte ‘On Location’ MatrixStore cluster that can be rented out through our dealer network. (Thanks to Ken at QLS)

Benefits:

- 2 copies of data protected at all times (4 disks would have to simultaneously fail for data to be unavailable)

- Secure from unauthorised access (audited access to content) 

- Cluster housed in a moveable, noise reduced, sealed XRack Pro enclosure.

- Replicate content to an offsite location (Internet access required) 

The View from OM

If you don’t have two copies of your data, you do not have it. If you cannot find your data, you do not have it. Reduce the risk of shooting on tapeless devices by making sure your content is protected and secure.

Please contact us for more details.

Anyway, press release:

PRESS RELEASE  

The MatrixStore archiving solution provides instant on-demand access to digital assets 

London, UK – Object Matrix, a developer of secure asset archive and retrieval software, has 

today partnered with distributor Channel Dynamics to launch The MatrixStore, a multi 

purpose disk based archive solution. The MatrixStore provides a secure archive to businesses 

with large data libraries at a fraction of the cost of a tape library. Ideal for the broadcast, 

design and creative industries, this solution offers users on-demand access to digital assets at 

a more affordable price point.  

Nick Pearce, sales and marketing director of Object Matrix, commented: “As the creative 

industry begins to rely more on rich media content in projects, storage requirements also 

become more advanced and demanding. With file based workflows becoming more common, 

the scalable nature of The MatrixStore makes it an ideal and cost effective alternative to tape 

libraries, while enhancing the workflow by delivering on demand access to archived content. 

Additional storage nodes can be added as the storage requirements increase, providing users 

with an easily expandable digital archive. Although it is not market specific, this product is 

particularly well suited to creative environments in which there are more requirements to 

provide access to current and legacy content.” 

As the demand for instant access to data rises and acquisition technology advances, the 

creative industry is increasingly moving towards a ‘tapeless’ workflow. The combination of 

inexpensive server hardware and powerful software provides users with an economical and 

easily expandable method of archiving all their assets within a project. Archived data is 

always available and the ease of installation and administration allows businesses 

to concentrate on their core competencies, rather than the demands of IT management. 

I will post images of the launch in another post.

The theme of the event is based the emergence of file based content and moving towards tapeless, commoditised workflows.

The seminar is being held in London on Thursday 27th November 2008. If you are interested in attending then please feel free to sign up here.

(From Wikipedia)

The growth rate, as can be seen, is very linear. As can be seen in the diagram, we’ve risen from a 1GB HDD to a 100GB HDD in 10 years, or a 10GB HDD to a 1TB HDD in… 10 years.

Thus, every 10 years storage capacity becomes approximately 100 times more energy efficient to use as an archive.

Meanwhile, the fall in price for a HDD has been huge, but exponential. Back in 1980 you’d be looking at a staggering $233,000 for a GB of storage space, where as today a GB costs less than 20 cents. Placing sample data of HDD prices from 1998 to 2004, I produced the following:

So, growth in disk size is linear, whilst the fall in disk $/GB exhibits exponential decay.

Overall, this makes for very good archiving economics on HDD.

Better Disk Electrical Usage Economies

The amount of electricity used by a disk drive (regardless of storage capacity) has also fallen, though nowhere near as dramatically as the capacity per HDD has risen.

Let’s take a modern HDD: e.g., the Hitachi 1TB Deskstar 7K1000.B gives “43% energy savings when disk is on standby”. That’s 5.2 watts on standy instead of 8.4 watts. Therefore, in archive usage, if a disk is idle for 700 hours in a month, it’ll use 3.64 kWh.

This compares favourably to most typical HDDs 8 years ago; e.g., the 30.7GB Maxtor, which drew 6.8 watts on standby and 12.9 watts when seeking. In archive scenarios, we calculate this as a 30% energy draw saving.

Whilst 30% energy efficiency savings are never to be turned away, the fact that the Hitachi stores 1TB of data (33x the amount of data) is far more significant. Our conclusion? In the long run, more important than paying a premium for the latest “green technology” is buying a solution that allows the modular addition of capacity; a solution that allows the use of the best available technology at the time of that addition.

Taking the trend of increased disk space in a HDD unit and falling energy usage, it can be seen that 100x energy efficiency is being achieved every ~9 years.

New Technologies: 2.5” Drives and SSDs

All general electricity usage improvements on 3.5” drives are good but perhaps more exciting are the new 2.5” drives coming out and SSD technology in general.

A 2.5” drive uses 1.4 watts for Read/Write (e.g. Fujitsu 500GB, Hitachi Travelstar 5K500.B). No word on their standby electrical usage – but even taking the worst case of 1.4 watts that’s a significant improvement over 3.5” drives sucking 5.2 watts on standby.

Plugging the figures for 2.5” drives vs 3.5” drives into our electricity usage formulas results in a slightly higher electricity cost for a 2.5” drives configuration. Why? Because at the moment they store only 50% of the data, and because therefore one ends up using more CPUs, RAID controllers, etc. However, it’s getting close, and it wont be long before 2.5” drives will provide better TCO archive economics. OM therefore predict that within 8 years we’ll have 100 times better energy efficiency from HDDs!

SSDs are a longer term prospect, but a very promising one. Their electricity draw is significantly lower again, and people like (whatever the reality behind it) the concept of non-moving parts. I’d personally be VERY cautious about this technology up front – I believe it’ll take many years before the reliability of SSD will be proven as a suitable long-term storage medium and until then many people will get stung by storage that gets corrupted over time.

Optimum Time Between Disk Recycling Taking Into Account Energy Usage

When should an old unit of storage be rolled out in favour of a (presumably) larger capacity storage unit?

Let’s take this to be the point where a years energy usage to keep the old unit running is greater than the cost to buy a similar amount of storage in a new technology.

Projecting this into the future:

1. If current trends continue then disk sizes will be 4 times current sizes in 3 years time.

2. Calculations show that a 100TB MatrixStore cluster costs in the region of £2,271 per annum in electricity (including cooling, HDDs, CPUs); this is based on an 8p per kWh price for electricity and 24TB nodes. The purchase cost of 100TB (that’s 220TB before Raid and Mirroring) is £120k.

If we then assume that electricity costs rise at 10% per annum, and that rack space is NOT of a premium, we end up with a scenario where the electricity cost for a 100TB of storage “crosses over” the cost for a similar amount of storage after approximately 7 or 8 years.

At that point the new 100TB archive will arrive in 1/25th of the space, and will use 1/25th of the electricity.

Now, since rack space generally IS of a premium, we project rolling out old hardware every 6 years to be a good ballpark time-frame. (NB: this does not take into account the cost of moving the data).

Alternative Solutions: Tape

Changing tact a little here, let’s briefly discuss why tape based solutions might not be as green as you think.

Tapes need to be stored at the correct temperature and often are physically very space consuming. However, the main “non-green” factor of tape archives comes in if they need to be rolled out in their entirety, due to any of the following reasons:

• because of hardware maintenance costs rise for systems that are outdated
• because of higher manpower cost related to maintaining old tape formats / sizes compared to newer solutions
• because new demands on data usage require an updated solution

(to name but a few)

We do not believe it is unfair to suggest that tape solutions often get rolled out every 5 years or so.

When it comes to rolling out a tape solution, it may well be required to move all their data from old tape formats to new tape formats; a labour intensive exercise that is neither “green” nor economically.

In comparing a tape archive to a HDD archive; the cost of transferring data at the end of the tapes lifespan may well exceed the energy cost of keeping that data on spinning disk (and therefore easy to transfer).

OM’s CONCLUSIONS

Based on trends from the first 30 years of HDDs:

• Disk capacity rises by approximately 158.5% per annum
• Electricity usage per GB falls a hundredfold every nine years
• Economically, the overriding most important aspect of a disk-based archive is being able to roll in/roll out new/old technology as appropriate
• Being able to take advantage of SSD/other emerging technologies is advantageous
• With economics typical in our example, hardware should be rolled out every 5-7 years – possibly sooner if rack space is of a premium or electricity prices rises greater than 10% pa
• Alternative solutions often have hidden energy and “green” costs related to manpower required and frequent technology form factor redundancy
• As the costs have fallen, new demands for HDD data storage and usage have come in and will continue to do so

Comparing keeping an archive on disk to the cost of doing the same 10 years ago, that cost has fallen rapidly in terms of hardware, manpower and electrical usage. There is no reason not to believe that trend will continue.

External Links

Wikipedia Disk Capacity Over Time
Indopedia History of the HDD
Dave’s NetApp Blog: Disk Is The New Tape—Flash Is The New Disk
Hitachi 1TB Deskstar 7K1000.B
30.7GB Maxtor