« The Best SD Cards | Main | 'Visible Identifiability' and 'Decoder Availability' »

Wednesday, 06 November 2019

Comments

...Microsoft says "the hard silica glass can withstand being boiled in hot water, baked in an oven, microwaved, flooded, scoured, demagnetized and other environmental threats."...

But will it shatter if dropped? :-)

A lot of vanity in all this data-recording stuff.

[Look on my Works, ye Mighty, and despair! --Mike]

Just read the article on Project Silica - very interesting. It's a bit ironic that digital movies are transferred to black and white film for long term storage and conversion back to colour. More ironic that the recovered movie is apparently not ideal since film can't record the 1's and 0's of the original recording.

I still use pretty old slow cards with c.24mp sensors - for example in my Fuji XPRO 2 I have a 16gb SDHC Class 10 which is so old that I had trouble reading these specs (I think that means 10 mb/s)

I don't shoot huge amounts of images and I always 'download'images via a card slot.

Never eally had an issue with speed, although if I needed to quickly show some wedding guests 500 images, maybe I'd be looking at it more closely

They didn't mention hitting it with a hammer :)

My WORA is Google Photos.

Fascinating. If CFexpress can hold half a terabyte of data, why make it a removable media? I can see future cameras with soldered memory that would simplify the construction of the camera (no flimsy doors, bad connections and so-so weather resistance). As an added advantage, no more losing those cards. You could still lose the camera but that seems less likely.

[I always thought the same, but it gets pointed out to me that memory fails, and if it's replaceable then you just replace it. If it's built in then at minimum it has to go in for professional repair. --Mike]

Mike, both of my Lumix S1 cameras have two slots, one for SD (UHSII) and the other, currently, for XQD. Panasonic have announced that a pending firmware upgrade will open those XQD cards to CFExpress cards as well. Already physically implemented and ready to go. But in the meantime you can use your choice of XQD or SD or both.

Having seen storage formats come and go, I much prefer storage media that is man readable.

The 1960 US Census is on mag tape, IBM last made the drives in 1965.

To digitize/convert the Apollo files they needed to come up with a replacement for the whale oil used lubricate the drives.

I am scanning my images and storing them on NAS/RAID drives, but I will never part with the film originals.

Has anybody seen my paper tape drive?

As someone who lives in the world of data transmitted on fiber optics, much at 100+ gigabits/sec and a single fiber can carry up to 128 individual signals at 100G - I keep track of many new developments. The MS Azure project is not the first like this. Almost 20 years ago, researchers stored a terabyte of data in a tiny plastic cube using holography. Unfortunately, it was never commercially feasible. But the storage of data on CDs and magnetic disks has steamed ahead - the original hard drive I had in a PC (I'll spare you details of the days when computers had tubes and 8K was a lot of storage) was 20MB and the size of a book. I have a decade old iPod with 60G magnetic disk drive that is the size of a deck of cards. My backup drives are a 3TB disk and a 1TB SSD - basically like a memory card.

But there are two things to consider about long-term storage. For magnetic or solid state memory, it can wear out. Have you heard about the memory in Tesla cars wearing out? https://futurism.com/the-byte/flash-memory-old-tesla

Secondly, there is the issue of there being equipment to read the data. A few years ago a project to recover old NASA data depended on finding several old racks of equipment that could read it (https://www.nasa.gov/topics/moonmars/features/LOIRP/) but the project died when the people working on it retired and no further funding was provided.

The MS Azure glass may endure, but will anyone be able to read it?

I have over 30 years of files and data backups but the older stuff is junk because there is no hardware or software to read it.

There have been hundreds of media used for data storage - see https://obsoletemedia.org/data/.

Want to archive a photo? Print it!

Project Silica looks like a God-send. It's too late to save so much of our musical history . . . .

https://www.nytimes.com/2019/06/11/magazine/universal-fire-master-recordings.html

It breaks my heart.

It'll be interesting to see if XQD/CFE/Whatever takes hold, though I have some doubts:

The number of new, non-phone cameras sold is declining each year and sales have yet to hit bottom. And how many of those new cameras selling today (or will sell in the coming years) use the new storage devices vs. the widely available (and cheap) SD cards? Maybe someone can provide numbers, but I would bet that a majority of new cameras still use SD cards. Plus, most older model cameras still in use are incompatible with the new technology. So unless the manufacturers need to sell only a small quantity each year for the new cards to be profitable, I don't see how they can last.

As for the glass storage device, all that technology won't help if, as others already noted, the glass holding your priceless movies will shatter when someone holding it trips and drops it!

Funny stuff, WORA. Back when I was a systems consultant I worked on a project for the IRS, a debtor collection system where you basically had to ignore at least 3 mailing before being added to the outbound call system. And for pretty much all of you know the government agencies that might contact you will NEVER call first. If you get a call it's about 110% a scam. Do not respond.

Anyway, the system we built base on bid requirements had a write only tape system. There was nothing in the requirements to ever read the tape. So we had a set of files that were being written and stored to never see the light of day again. It was a interesting time in just how much money our government seems to waste on a consistent basis.

The Project Silica wafers look a bit like memory cards used in the 1960s Star Trek series. Every now and then Spock would slip one into his science console.

It is not enough that data can be stored in an eternally stable form. The data reader, its software, and its power supply needs to be eternally stable and available as well. Who today maintains a reader for old style floppy discs or computer punch cards?
Mike,perhaps nothing is truly archived unless it is directly human eye readable like your negative and contact print files.

Compare the HAL 2001 photo with the Microsoft photo. Mr. Clarke seems to have been a bit clairvoyant…

H.A.L. 9000 - H-A-L is derived from Heuristic ALgorithmic computer”. Clarke more directly addressed this issue in his book The Lost Worlds of 2001.

"I'm sorry Dave, I'm afraid I can't do that."

">https://encrypted-tbn0.gstatic.com/images?q=tbn%3AANd9GcTGapeGFS2djZ08BhgjyKnhkedZsYmt6OO1Y2enMdP6y-zuZoko">

It's worrisome that we can't access data on 5" floppies anymore or the smaller 3.5" ones, either. CDs and DVDs are not much longer for this world – who still uses them for music or film storage? Formats change so rapidly these days. I suppose the digital era is still in its infancy. How many more formats will come and go? Will there ever be an ultimate format?

Will we be able to access any of these formats in the time span they're crowing about? Will we still have electricity? I'm thinking of the 10000 year clock project here. (https://www.10000yearclock.net/index.html) Will people (or whatever beings may be around so far in the future) really be interested in us?

If your WORA is anything involving Google or some other company providing storage over the internet, you should probably look up MySpace. It may seem like Google will be here forever: they won't. And even before they go away, they may just decide that whatever service you are relying on is no longer important to them and turn it off, particularly if you are mot paying for it.

More generally if your WORA is a service rather than a collection of physical objects, it isn't a WORA. Especially if it involves Googlebook.

CFExpress card are already available:

https://www.amazon.es/Tarjeta-SanDisk-Extreme-CFexpress-1700-MB/dp/B07WVDV3FG/ref=sr_1_15?__mk_es_ES=%C3%85M%C3%85%C5%BD%C3%95%C3%91&crid=3MCFJDJ4CA1O8&keywords=tarjeta+xqd&qid=1573126313&sprefix=%2Caps%2C324&sr=8-15

What are you waiting for, Nikon?

We're slowly inching our way from the era of "big data" to an era of "way too much frigging data". We need curatorial AI.

I bought an XQD card for my Lumix S1 recently, and it's fast. It's also expensive, because Sony controls all aspects. So I was interested to notice that the S1H video model uses only UHS II slots and SD cards, which can also be fast, but are less expensive. And the same for the Leica SL2, which was just announced. The CFExpress containers replace small hard disks for mass video capture, which can run at several gigabytes per minute and over 29 minutes of shooting in video cameras (still cameras, for obscure reasons, must stop at 29 minutes). So personally, I can wait a bit for XQD.

Mike J. wrote, ". . . my former WORA technology . . . I haven't touched it in better than 20 years and all the image information is right there awaiting potential excavation. This method isn't practical any more — once it reaches this stage it's stable, but it takes too much work and cost to get to this stage."

AND

"Project Silica isn't currently intended for home storage . . . But if it turns out to be practical . . . it could eliminate the eternal headache of "write once, then back up over and over . . . at regular intervals until you run out of money or until some idiot drops the ball, at which point everything goes to hell pretty damned quickly," i.e., the current state of digital archiving."

My questions are these: Have you spent enough money and time backing up your digital photos to still make the "old-fashioned" contact sheets and negatives "too much work and cost"? Have you lost any digital photos because of failed storage drives or other "glitches"?

If hard silica glass is the future, I hope they test it (as mentioned by other commenters) when someone steps on the glass on the floor (wearing hard-sole shoes) -- or drops something hard on the glass. How does the glass fare if it is dropped on a corner? Accidents do happen.

Speaking of revolutionary new tech…PBS Frontline aired an interesting episode on AI the other night that’s worth viewing. It’s available for free on their website. Today’s crazy speed and processing power is driving AI and the recent advances in AI’s visual task ability (cameras/sensors) is both interesting and creepy.

While Frontline tends to cast an ominous tone over everything they cover I think there is plenty to be concerned about with AI. It will be interesting to see how AI shakes out over the next decade. It’s just another tool after all and I imagine its impact will depend in large part on those who wield it. This is the part that worries me though. The lack of ethics in business and government these days is worrisome.

I'd probably own a Z camera if not for the XQD card. $90 for a 32 GB card?

The Z50 uses SD so anyone wanting to carry along one to compliment or backup their Z6/7 needs 2 storage formats.

Every camera I have is SD. My laptop, desktop and almost every PC out there can read one. Why complicate my life?

Sometimes new tech is simply not worth the headache. If old tech is sufficient, cheaper and easier why change?

As far as I am concerned, it remains to be seen where CFExpress will take off as a standard format, much as the original Compact Flash format did back in the day.

CFExpress was developed jointly by Sony, Nikon and Sandisk, but Sandisk has yet to put any resources into manufacturing CFExpress cards, and my guess is this is once again due to Sony's Machiavellian IP practices. Where oh where is Sony Beta?

With respect to actual applications requiring CFExpress, I see this format as more suitable for video applications than stills photography, with the recent release of 8K-capable video cameras.

With respect to stills, the main application for CFExpress would be for shooting sports, and despite the view from YouTube video reviewers who are constantly hyping faster frame rates (e.g., 14 FPS, 20FPS and higher) for new cameras, the fact is these guys are amateurs when it comes to shooting professionally accredited sports.

So here's some context for the gang for consideration: 8 FPS is plenty for shooting professional sports, 11 FPS is just icing on the cake. Most pros shoot in this range of frame-rates, and some even turn the frame rates down to 5-6 FPS.

So, 300 MB/sec UHS-II SD or XQD cards are fine for these applications.

Why? Because sports shooters, unlike amateur YouTube reviewers, shoot like a Navy Seal or Delta Operator: 2-3 "taps" on the shutter when capturing peak action. Its their decades of experience that lets them do this with accuracy and precision. There are always use-cases that require faster frame rates or more frames, but these use-cases are typically outliers and non-standard shooting situations.

All that shooting at 14 or 20 FPS means is: a whole lot more frames in your take you have to cull down to a small set of selects to find the one hero shot that will get published. All this culling takes time away from getting your hero shot sent out in time for deadline press.

The other thing that shooting at 14 - 20 FPS, etc. does is...unecessarily burn up shutter life. And believe me, you don't want to have a shutter go south on you in the middle of a big pro event, as happened to me when shooting a MotoGP in 2007. Ask any pro sports shooter, shutter life is typically one of their very top 5 requirements along with....Durability and Reliability.

We keep getting these claims that X or Y storage medium will solve the digital preservation problem because the storage is permanent, or good for 100 years or so, whatever. It's certainly true that longer-lived storage media are better than shorter-lived ones. But in general the problem soon becomes whether you still have the ability to read the storage medium. There's stacks of media lying around that are pretty readable except that we don't have workable devices to read them. The pressure that you mention... the next device will be "mo' bettah" and will replace the earlier device, specially for less mainstream media.

The digital preservation problem isn't really about the storage medium, it's mostly about continuing care and continuing money. It's generally not a problem to move information from one storage medium to another... while you still have access to the old devices... so you have to have someone there to do it at the right time.

[Come to that, I don't currently have a darkroom or a film scanner, so my old "WORA" system isn't exactly accessible either. At least not immediately from here. --Mike]

Syylex AG in Villingen Germany in 2011 invented a glass DVD called the GlassMasterDisc that apparently had a lifespan four times longer than the M-Disc. It was terribly expensive at 160 euro and the company went out of business in 2015.

The "A" in WORA stands for "anywhere," not "always"! The term is not used in the field of digital preservation. A glass storage medium might be durable and more disaster-ready, but remember that you need a lot of machinery and software to actually read it that is more precarious. Human-readable media like film is better when it comes to that.

If anyone wants an intro to digital preservation, I recommend listening to "Preserve This Podcast" from the Metropolitan New York Library Council.

https://preservethispodcast.org

The biggest problem to broader adoption of that Project Silica tech is the writing lasers; the kind of femtosecond laser required for permanently writing index changes into glass is gonna be big, expensive, at least a bit fussy, and sorta dangerous. (As is sometimes said among the high-power laser jocks I've worked with: Do not look directly into the laser with your remaining eye!) I think the tech will be a while in getting down to computer-tower size.

That said, it seems like a fantastic archival storage medium, far better than anything else I've ever heard of. I can definitely see it being broadly adopted by people looking to archive huge quantities of data.

I have discussed this sort of thing a few times out there on the web. My experience is from two companies that have governments requiring access to data used to design "things". (Airframes and Health Care devices)
At the airframe company, we had to reload some 18bpi tape. Once the company found a tape drive and got it up and running they mounted the old tapes. They did not snap, as expected, but the oxide separated from the tape in a cloud of brown dust.
This same company was using Optical drives (glass substrate) that looked like CD's in their attempt to have manuals stored around the world. A master set in the US, Australia and somewhere in Europe. I don't know if they are still there, but it was when I left in 1999.
Now, here is the question for interfaces. Will computers be able to read ATA drives, USB drives (some company's have the "courage" to stop using USB ports), SATA, SAS or what ever flavor of the week is popular today. I just went through my old CD's from the early 2000's and they still worked, I am not sure about the ZIP drive and disks I have on the shelf.

The nice “feature” of using rotating hard drives for archival storage is that they usually fail pretty quickly - i.e., a single-digit number of years. If you’re keeping dual archival copies, you then create a new archive using then-current media from your surviving archive. This automatically (lol) ensures that your archive remains readable using current technology.

The comments to this entry are closed.