I'm happy to report that SanDisk RescuePRO Deluxe worked like a charm, in the end, on my accidentally overwritten SD card.
To recap: I had a few travel shots I wanted on a 32 GB, 30 MB/s SanDisk Ultra card, but when I did the speed tests on the new UHS-II card the other day, I reformatted and made new shots on the beginning of the older Ultra card not once but several times—before downloading my latest shooting. Oops.
When I reported this, readers recommended data recovery apps in a quick flood of comments. After checking several of the apps out, I decided I could really only test one, so I chose the app made by the maker of the card, and bought the "Deluxe" version; my reasoning was that if I was going to fail, it would be better to give the app its best chance, and fail with the turbocharged luxury version.
Note: TOP Tech. Ed. Ctein tells me that different toolz have different skillz. So if one recovery app doesn't work for you, another might...even on the same lost data. (He also noted that it's a good idea to lock the card you're recovering data from before you insert it in the card reader or SD slot, which I hadn't done.)
At first the app didn't seem like it was working. It appeared to make progress for a little while and then choke. So I read the instructions, and watched a YouTube video showing how to proceed, and started over again. The progress report seemed wonky—for "time remaining" it was giving me numbers as high as 2,000 minutes, and progress was stuck on 34% for well over twelve hours. All in all the program took about 36 hours to run on my laptop (an early 2014, 13" MacBook Air 1.7 GHz core i7 with 8 GB RAM and a 512 GB SSD). After seeming stuck for a day or so, the "time remaining" began to plummet and the remaining progress percentage rose steadily until...
...Voilà. I got back all the pictures I wanted...
...Most of which were of this strange atmospheric phenomenon I saw out the airplane window. Ctein, many of whose best photographs in my opinion are of the atmosphere or what's beyond it, informs me that this is called a "glory"—a full circle rainbow of restricted circumference that sometimes happens when you look into water vapor from the exact same direction the sunlight is coming from. Glories (they're sometimes called "the glory of the pilot" because they're most often observed from planes) have an angular size of 5° to 20°. That depends on the size of the water droplets in the clouds. The shadow of the airplane often appears inside the glory, as here, and the glory "follows" the plane like its shadow.
Ctein says, "the physics of it is actually pretty complicated, and interesting if you find physics interesting."
The SanDisk RescuePRO Deluxe recovered a great deal from the card, I must say—not only what I was looking for, but at least most of the last six or seven shooting sessions I used the card for, going back more than a month and half. Note that the pictures I wanted were the first forty or so shots on a "clean" (formatted) card, and that I had reformatted the card and reshot at least three batches of more than 50 shots on top of the pictures I wanted. The recovery app got them back anyway.
As for the glory, it seemed more vivid to the eye than to the camera; the plane's windows were unfortunately both dirty and noticeably scratched up and damaged. It was one of those old jets that had a cloth label stitched to the back side of the headrests that read "Seat bottom cushion may be used as a flotation device." As I quipped to my girlfriend, I'm glad I didn't have to rely on it for that, as it wasn't even working very well as a cushion.
Wikipedia says, "In China, [the glory] is called Buddha's light. It was often observed on cloud-shrouded high mountains such as Huangshan Mountains and Mount Emei. Records of the phenomenon at Mount Emei date back to A.D. 63. The colorful halo always surrounds the observer's own shadow, and thus was often taken to show the observer's personal enlightenment." I'd say if you need a sign in the clouds to signal your enlightenment you're probably not enlightened, but seeing your own shadow inside a glory must be glorious indeed; the unusual sight does seem tinged with a touch of magic, and must have seemed more so before the age of traveling above the clouds made it more common.
Mike
(Thanks to Ctein)
Original contents copyright 2015 by Michael C. Johnston and/or the bylined author. All Rights Reserved. Links in this post may be to our affiliates; sales through affiliate links may benefit this site.
(To see all the comments, click on the "Comments" link below.)
Featured Comments from:
Matt: "The reason your shots, and many others, were still around, is a thing called 'wear levelling' which basically makes the card overwrite the oldest files on the card first. The memory only has a certain number of uses, so without this technique, the cards wouldn't last nearly as long.
"This is done on the bit level, not on the level of files, but the effect is the same, since we're not randomly flipping bits in files, but writing large blocks of information at the same time."
Arne Croell: "The glory is also known as the spectre of the Brocken. The Brocken is a peak in the Harz mountains in Germany, which is often shrouded in fog; it is also known as the mountain where the Witches Sabbath happened in Goethe's Faust."
Jim in Denver: "Glory (and I'm a jet pilot professionally, so I'm more than somewhat familiar with the phenomenon).... Essentially the airplane is the 'lens,' not dissimilar from a drop of water, and that makes a rainbow directly in-line with the shadow of the airplane. The cloud is just something to see it projected on. The glory gets bigger and smaller depending on the distance you are from the cloud. It only exists (viewing) with your back to the sun. They actually happen without clouds, but are much harder to see—but next time you are coming into land, it's a clear,sunny day, and you are on the down-sun side of the airplane, you can sometimes see it on the landscape rushing by as you get fairly close to the ground."
Now you can also see why secure delete programs exist.
Posted by: Doug Nelson | Wednesday, 21 January 2015 at 09:50 AM
I believe the phenomenon is called a Brocken Spectre, commonly observed by climbers and mountaineers. It is named after a German mountain called the Brocken where the effect was first noticed.
Posted by: David Paterson | Wednesday, 21 January 2015 at 10:06 AM
I think that what you are seeing at work is the wear-leveling built into the controller on the SD card. Flash memory can only be written to a certain number of times (in the thousands to hundreds of thousands, depending on the technology used), so the controller on the cards is designed to even out where on the card your data is stored. With such a large card, if you don't fill it up each time before you format, the old data hidden by the new format, but can still be there (note that this is why when you sell computers, flash drives, and even hard drives you should run a utility that writes random data and erases it). If the card didn't do this, you would always write to the same area of the card and wear it out quickly (every time you write or delete a file, the table that stores the file information is updated).
Good that you got the images back. Very interesting phenomenon too.
Posted by: James Ziegler | Wednesday, 21 January 2015 at 10:18 AM
The reason you got your older pictures back is because the format doesn't really do a TRUE format. It just cleans out the index to all the photos. Pretty much what a delete all does.
A true format rewrites each sector of the card. I don't know what these modern memory cards actually do during a TRUE format (Which you cannot do.) but if it really did a true format you wouldn't be able to get anything back.
It's the same for your hard drives. The drive is given a true format at the factory where it finds defective sectors and uses spare sectors as replacements. A quick format just resets the index. So whenever you do format a hard drive just do a quick format and you can then recover your data if you formated the wrong drive.
Security experts tell you to physically destroy your disk drives for this exact reason. Or use a powerful electric magnet on the thing.
So if you do a format or delete all then use the recovery utility and get them all back.
That being said I haven't worked on this stuff for many, many years so things could have changed but I don't think so.
Posted by: John Krill | Wednesday, 21 January 2015 at 10:25 AM
I'm glad you succeeded.
Those circular rainbows can be seen in the sky occasionally while walking around the Bay Area. Not common, but not rare either. It must be the fog, which is often lurking somewhere nearby, providing water vapor that gets blown into the sunny sky.
Posted by: Scott L. | Wednesday, 21 January 2015 at 10:34 AM
If there were no one present to see the glory, would it be there?
Not the tree falling in the wilderness question. Is the obstruction to the light causing the phenomenon?
Posted by: Michael Matthews | Wednesday, 21 January 2015 at 10:43 AM
Btw, a good introduction into such phenomena is "Color and light in nature" by David Lynch and William Livingston. Right now, Amazon has only used versions, but reprints are apparently available from the author: http://www.colorandlightinnature.com/
Posted by: Arne Croell | Wednesday, 21 January 2015 at 10:47 AM
I think you benefited from a difference in how space is managed on these cards versus hard drives. Hard drives always start writing in the first available space. Delete a file, and it's likely to be overwritten by the next file. Cards don't.
The way I understand it, the card controller spreads the activity to all areas of the card to maximize the life expectancy. If you had completely filled the card after reformatting, I think your story would have had a different ending.
I hope someone who understands this better will fill us all in.
Posted by: Omar | Wednesday, 21 January 2015 at 11:47 AM
Mike, I'm guessing you know this, but the essential workflow for out of airplane window shots is to crank the contrast slider up pretty high - which overcomes the loss of contrast from atmospheric haze, double windows and dirt. We unconsciously adjust for for the low contrast when we are looking out the window.
Posted by: Alan Fairley | Wednesday, 21 January 2015 at 12:28 PM
I follow the site silently, absorbing as much information as my hobbyist background can manage. I understand the frustration of data loss much better than exposure, tonality, etc, in photography, and I was so glad to learn you were able to retrieve the images.
Posted by: Dave Harrington | Wednesday, 21 January 2015 at 02:20 PM
Here is an excellent site for atmospheric optics (and Glories).
A recent entry on my web site includes a photo of a Glory and Brocken Spectre inside the Grand Canyon.
Posted by: DavidB | Wednesday, 21 January 2015 at 03:33 PM
Dear Michael,
A fair question.
There are diffractive effects that occur around the shdow, but they are subtle and wouldn't be visible in a photo like this. The glory is entirely the result of the light bouncing around in the fog droplets-- it'll be there even if the observer is very small (or absent).
pax / Ctein
Posted by: ctein | Wednesday, 21 January 2015 at 10:05 PM
Usually when you see that halo around your shadow it's a strong indicator of icing conditions. My pilot instincts have me reaching up to turn on the engine and wing anti-ice.
Posted by: Dave | Thursday, 22 January 2015 at 12:12 AM
Mike, your account of "Buddha's light" is accurate, according to the in-house discussion of the phenomenon in Buddhist circles in China. Mt. Emei is the Buddhist mountain in Sichuan Province and Mt. Huang is usually identified as a Daoist mountain. On the top of Mt. Emei the corona/spectrum/halo surrounding your own shadow projected onto misty clouds has been a startling and ethereal sight since the Han Dynasty, as you report. Do the Daoists on Mt. Huang witnessing the phenomenon call it "Buddha's Light?" I think not, sectarian rivalry having been a part of the Chinese religious scene since the Han Dynasty as well.
Posted by: Rev. Heng Sure | Thursday, 22 January 2015 at 03:30 AM
I recommend PhotoRec, which is both free and open source. I have only needed it a couple times, but it has worked perfectly.
Posted by: Robin Parmar | Thursday, 22 January 2015 at 10:57 AM
http://www.nature.com/scientificamerican/journal/v306/n1/full/scientificamerican0112-68.html
Posted by: Hélcio J. Tagliolatto | Saturday, 24 January 2015 at 03:49 PM
Mike, my first reaction when I saw your post was: oh! he's used one of the pictures I've sent him to illustrate his post! That lasted only for a split second, of course, but I did consider sending you my shot of this amazing phenomenon for the Keck Observatory contest. As you must know, I ended up chosing other images to send you but that was an eerie feeling for sure.
Oh and glad to hear your valued shots are safe!
Posted by: Thomas | Sunday, 25 January 2015 at 07:11 AM