I've always welcomed comments on this site, and sometimes the comments, collectively, are better than the original post, but I seldom simply toss up a subject and open the floor to other speakers, depending on commenters to provide the information and interest. But I think I'll try that now. The occasion is a question from an "old pro," Scott S., that I can't answer:
I read your site regularly and enjoy it immensely. I have been a pro for 25 years, working mostly for non-profits.
I am faced with a storage issue. I have strung together pairs of hard drives (6 sets totaling about 7 TB to date) that serve well for short-term storage. What do most people do for a long-term solution? I have heard so many options, I am just confused. Do you know a site or link that may help?
What about this? What does the future of archiving look like for digital images? The archiving of prints, slides, negatives, and other traditional media, a sub-specialty of paper conservation, has a well-established science and standardized practices. But what can you do now, or what do we need to achieve in the near future, for long-term, permanent archiving of our digital media?
Does anyone know what actual archiving institutions are doing? Is anyone aware of the state of standard practices among conservators?
Featured Comment by Craig Norris: "This subject has been actively on my agenda for the past ten years, ever since it became my responsibility to work on the problem for TV stations (my other career outside photography). Video production typically consumes up to 70 gigabytes for every hour of high definition video that is recorded in some form of compressed file. Even with low-end video formats like HDV, using much deeper degrees of compression, it still requires something in the order of 11GB for each hour of recorded material.
"In still photography the size of our problem is some order of magnitudes smaller in terms of absolute data storage consumed, but the requirement to faithfully preserve the data indefinitely is no less important and no less difficult.
"To get straight to the question posed by Scott, I can pass on the fact that TV stations today are already using or looking to use LTO data tapes as the main digital bulk storage medium. The current version called LTO-4 has a native capacity of 800GB per tape cartridge.
"Linear data tape cartridges aren't good for day-to-day random access, so the common practice is to keep the active material on RAID hard disk arrays, and to back it up regularly to LTO-4 tape, which is usually housed in robotic auto-changing libraries containing hundreds or even thousands of tape cartridges. Next year will see the introduction of LTO-5 tape, with double the capacity of LTO-4.
"The above-mentioned solution being adopted by TV stations isn't economically viable for the average photographer, except for the fact that a standalone LTO-4 tape drive can be used to back-up the multi-terabyte hard disk storage attached to one's workstation.
"According to my latest research on behalf of TV stations, and in the quest to solve my own data storage problem at my studio, blu-ray disc is emerging as a practical data storage medium. With the new dual-layer version offering 50GB of capacity per disc, and an estimated data shelf life in the order of twenty years, blu-ray disc is likely to be the next data archival medium that I implement for my digital photo archives.
"Supporting my comments above is the observation I made a few days ago when I was hired to photograph the NBC Universal Olympics facilities in Beijing. I saw NBC recording a very large proportion of the Games broadcast on Sony's XDCAM HD format discs, a close cousin of the Blu-ray data disc."
Featured Comment by Stephen Barnes: "Information Management is my day job, and the one thing I'll say is that it's just as confusing for the professional archivists as it is for the photographers.
"I've worked with the National Archives for Scotland, Public Records Office for Northern Ireland, and The National Archives in England, and none of them have any consistent method or approach.
"The first problem is archive longevity. Any images we needed to send to them were put onto MAM-E [MAM-A in America] Gold DVD disks which have an estimated 200 year retention period. I personally use Kodak Professional Archival DVDs. All images are then copied onto a SAN which retains multiple copies of each file on-line. Finally, this SAN is backed up to tape. We therefore have three different technologies on which images are retained.
"If a technology looks like it is going to be superceded, we transfer all data to the new medium as soon as possible, to make sure we have the ability to read the data. Old technologies (eg. floppy disks, even VHS!) are archived along with a number of working devices. (There's no point in archiving something if you don't know if you'll have a device in the future!)
"The same is true for formats. While it doesn't look like JPG/TIFF are going to go away we would convert to alternative formats (PNG, PDF, DNG etc) to minimise risk.
"What this all means is that for sensitive and historically important information we have up to four copies of the same data on each of three media. That's 12 copies of our images.
"But the professional archivists have a preferred media for long term preservation—silver halide prints."
Featured Comment by David A. Goldfarb: "If one wants to preserve images for the ages and not rely on data transfer, I agree that the solution is to make the best archival prints one can make and store them according to the accepted archival practices for paper media. An important principle of archiving is that a long-term archive should be 'human readable' if possible.
"I met a historian once who had responsibility for the archive of an important figure in U.S. foreign policy. He had an office full of 8" floppies, the original computer used to produce them, and a huge IBM printer the size of a coffee table that could print files from this computer. Sometimes that's what it takes to preserve something worth preserving in digital form. I suspect by now, if he hasn't migrated the data digitally, he's probably printed it all, and scanned and/or microfilmed it."
Featured Comment by Charlie H: "Personally, I am all about the photo-album. Last night we pulled out a few clunker sized albums of my wife's trip to China twenty years ago. Pics looked great, the kids were all over it. We haven't so much as lifted a finger preserving this archive. Barring a fire, they should be around for a good while and that's enough for me. Long live the photo album."
Mike replies: Charlie, you stole my thunder. I'm writing a column now making the case for the lowly album as a surprisingly good alternative for some people and some purposes. It's interesting to say the least. And it feeds into what Gordon says below, too.
Featured Comment by Gordon Lewis: "I don't mean this to sound gloomy, but if you look at the history of the world as we know it, nothing lasts forever, at least not in its original form. The best we can hope for is that our work outlasts us. Once we're gone and it's in someone else's hands, who knows? They may value it and want to preserve it; then again, they may not—and this assumes they know where it is and want to go to the trouble of examining it.
"A previous poster mentioned that there are short-term and long-term dimensions to storage. I would add that there are commercial and non-commercial dimensions as well. You can't sell what you can't find or can't read, so there's a strong financial imperative to preserving your work. Family and personal photos, though non-commercial in intent, could be invaluable to the owner, yet it's amazing how little time, money and effort the average person puts into preserving them.
"For my own part, I've decided that instead of simply preserving the photos I've taken of family and friends, I'm going to print them and distribute them as albums; to spread the wealth, so to speak. It's possible they'll last longer or at least be better appreciated if they're distributed rather than hoarded.
"I'm not lecturing anyone else, mind you; just sharing a few reflections. The way I'd summarize my feeling is this: Save your work as best you can, but also remember who you're saving it for and make them a part of the process; otherwise, your preservation efforts may end when you do."