Column 100: Looking Forward, Backward, and Sideways

Introduction: I'd like to congratulate Ctein on his 100th post for The Online Photographer! For four decades, Ctein has been one of the world's leading technical authors in photography—and one of the few technical experts whose work is based so securely on firsthand experimentation rather than only  "readin' and regurgitatin'." We're lucky to have the benefit of his expertise here, and I'm grateful for his help and friendship. Here's to the past 100, and to the next!  —Mike

By Ctein

When Mike asked me if I'd write a regular column for him, I had no idea if I'd be able to sustain it. Three column ideas each month, especially given my penchant for writing long? I've always envied newspaper columnists and cartoonists. While I can aperiodically write great columns and come up with pretty damn funny cartoons, my well would run dry mighty quickly if I had to do it every day.

Yet here we are, at Column 100, and I have a backlog of several dozen ideas that ain't gonna  go away unless Mike decides he wants me at least weekly. I'm amazed. Column 100, almost the third birthday. (By the way, I'm really looking forward to the cake, chocolate being preferred. And the surprise party.)

Old news about tomorrow
Eamon's recent column on "Alternative Futures" made me realize I had more to say on the subject of prognostication, an appropriate choice for the Centennial column. I've already written a couple of columns about this:

"A Brief History of Electronic Photography"

"The Shape of Things That Came"

I've been  one of those industrial consultants who's been paid quite well to predict the future. It's like picking stocks or handicapping ponies: it's easy to make fun of our truly boneheaded mistakes, but the real business is based on percentages. We don't have to be right all, or even half the time; we only have to be right a lot more often than average to make us (or rather, our clients) money.

I've made great predictions (reasonably accurate ones about DSLR designs and time frames 25 years before the products came to market) and truly lousy ones. The lousy ones are more useful; if we don't learn from our own mistakes, someone else will, and then they'll be better at the game.

In the mid-1980s, almost all knowledgeable consultants predicted the demise of rotating-disk hard drives. The dumb ones said that solid state memory would supplant hard drives in 15 years. The smart said 10–12. I was "smart." Yet, here I am 25 years later, surrounded by gently whirring spindles. Who ordered that?!

Solid-state storage prices are about where I'd have expected them to be, circa $1/GB ('80s dollars), in contrast to hundreds of dollars per megabyte back then. That part's right. What's wrong is that we couldn't imagine that disc manufacturers would be able to drive the production costs on a precise and complicated mechanical device down to 3 cents/GB ('80s cents). Storage costs have dropped faster than RAM costs, year after year, and the end is not in sight.

Old news.

The X Market
Here's where it gets interesting. Our prediction could've still been right. There is a certain overhead to manufacturing and selling that it's hard to undercut. That's why you see terabyte hard drives for $80 (2009 dollars) but not 10 GB drives for 80 cents. In 1985, you could reasonably predict that people would want several gigabytes of computer storage today. That would be hundreds of times what they had back then, but it was a plausible extension of existing trends and patterns, and it would make a hard disk business unsustainable, no matter how little a GB of disk space cost.

What we didn't predict was how cheap storage (and other computer-ish) improvements would change the market. That's called the "X Market"—it's all about how the product you're introducing changes the marketplace you're selling into rather than how much of the existing market you'll be able to capture.

We were sure this would be obsolete by now! Blame the X Market.

Had it not been for the X Market, hard drives would be specialty items today. Instead we have the Web, ubiquitous digital cameras (with pixel counts considerably larger than we expected most people would demand), digitized and downloadable music, and home video and motion picture storage. Many consumers feel cramped with 100 GB of storage. Home entertainment systems are available with over 10 TB of storage. A consequence of super-cheap storage, at the same time that it's what makes said storage a viable product.

That's the part of futurism that's hard to predict. It's what keeps life interesting.

Ctein

Send this post to a friend

Terabytes

Featured Comment by Carl Blesch: "Here's another way I like to look at storage capacity and illustrate the concept of logarithmic growth in a way mortals can conceive. I was able to put all of my saved workplace computer files from 1986 (the dawn of the PC age at my company) through 1997—just over a decade—on a single 650 MB CD-R. My saved files from 1998 and 1999 fit on another CD-R (thanks, Microsoft fatware—especially for the PowerPoints with lavish illustration and animation). Then it took one CD-R per year until I left that job in 2002. Shortly thereafter, I took up digital photography (scanned film at first) and sound recording. The next year, I scanned an image on 120 film at a 16-bit depth and 4000 dpi, and it took the whole CD-R to accommodate that one TIFF. Similarly, a single CD-R held only one hour's worth of audio I recorded in AIFF. So the contrast: one CD for a decade of my life's work, vs. one for a high quality image or hour of high quality sound. Fascinating!"

Comments

[Riley]

It just seems like yesterday that we had to partition a forty megabyte drive since MS-DOS could only address 32MB.

[David Dyer-Bennet]

Optical storage was going to replace magnetic storage, too. I learned that from Datamation in about 1970, and again every 5 years since then in one source or another. And we did finally get ubiquitous optical storage -- but just as an archive and transfer medium, so far. I suspect this was caused by the same thing as rotating magnetic remaining around even though solid-state met its goals; the price of rotating magnetic dropped immensely more than people anticipated, and the demand for large storage is higher than anticipated.

[Jim in Denver]

I remember my first hard drive in the '80s. 40MB, big for the day. I did everything I could to fill it up, and I recall that I made it just a bit past 22MB...

Now, just like many people, I am bummed that I can't use anything bigger than 500GB in my laptop! That is, actually, quite insane, but most people just call it progress...

If you didn't see it before, the link below is a must-watch. Sometimes the comedians can say the things we can't, or don't know how.

http://theonlinephotographer.typepad.com/the_online_photographer/2009/04/everythings-amazing-nobodys-happy.html

[Chris]

"And the surprise party."

We had that. It was fun.

[Peter]

Very well said. Although I am technophile at heart, I do not envy the futurists one iota at all. I look forward to hearing that huge backlog of posts. Don't try to go weekly though Ctein. Three a month from you is probably about what my poor brain can handle comfortably.

[Miserere]

Ctein, why don't you predict something useful, like the winning lottery numbers? ;-)

Congrats on 100! I'll be sticking around for the next 100, and predict I will be reading #200 with my eyes closed with a cable connected to my cortex feeding your words and images directly to my brain. Looking forward to that!

[Wilhelm]

Congratulations!
I always thoroughly enjoy your columns, even the ones with which I disagree.
"May you live long and prosper."

[Tim]

News just in...

SSDs are SO yesterday

Meet HRD (Hard Rectangular Drive):

http://www.theregister.co.uk/2009/06/15/dataslide_berkeleydb/

[kevin]

The classic future that never arrived had flying cars. I'm still waiting for mine, but I's settle for a single-stage-to-orbit spacecraft.

Oh, and robots were going to make breakfast for us.

[Ed Kirkpatrick]

All this cheap storage is great and I have lots of it, but we all still have the problem of how and where to back up all that data onto something that is stable... What we really need is nearly twice or three times even the back-up space as data files. Can;' put all that data on floppy discs anymore... :)

[Scott W]

I've been bemoaning the slowness of read/write and data transfer speeds for several years now, ever since my image files grew into double-digit MB's and my imagemaking grew into 4 GB per hour shooting. It simply takes too long to move data from one hard drive to another. That's really limiting the viability of multi-TB networked drives, in my opinion.

[Tom]

I thought you would like to see my predictions for film for 2010.
http://thephotofather.com/?p=412

[bl]

In the early ninties, when a terabyte of data could only be held in a large climate controlled room full of refrigerator-sized boxes, I was at an IBM conference where the keynote speaker, head of the IBM storage lab in Palo Alto, held up a matchbook and proclaimed that their goal was to fit a gigabyte of data on a device no larger than the matches in his hand. It was the stuff of fantasy.

CF cards are still a source of wonder for me.

[Mike]

I remember my first really big disk was a $1000 Connor 210MB unit. I've recently installed 2 1TB units for under $300. That's progress. Just made a composite photo — 300MB. That was impossible not too long ago.

[Jon Bloom]

The first hard disk I had was 10 MB. Now I have a camera which produces an image file that would exceed the capacity of that disk! I'm about to buy a new laptop with a 500 GB disk drive -- and I'm wondering if it will be enough.

[thalarctos]

When a technology is entrenched, there is a strong incentive to squeeze everything possible from it. Disc drives, as you mention, are one instance of this--tricks like perpendicular magnetic recording and the use of giant magnetoresistance in read heads keep moving disc drive technology forward.

Other examples can be found: 15-20 years ago, the smart money was saying that copper wiring for computer communication was already obsolete--yet now we have gigabit ethernet over copper twisted-pair wiring. Likewise, it was thought that the wavelength of UV light would limit the development of silicon chips--but the use of immersive optics has allowed feature size to push down to below 30 nm. Maglev trains have been the Next Big Thing for thirty years, but instead we have old-fashioned steel wheels on steel rails going 300+ km/h on a daily basis.

But I'm still waiting for my personal jetpack. The future just ain't what it used to be.

[Andrew]

The future is always a few years away. In the early 80's I used to visit the UK atomic energy authority on occasion (To use their Cray) and at that time they were predicting Nuclear fusion to generate electricity in 30-40 years. Now they are predicting this will happen in 30-50 years. hmmm...

[Dennis Ng]

@Carl: So true what you say. My wife said that my life is now dedicated to file copy instead of doing anything useful, not that she will say that taking photo is that useful hobby but way better than file copy. I went back to slide taking for snapshot and look at them as it is to avoid the copy, backup, organise, ... cycle and taking too many pictures.

[Mani Sitaraman]

Happy 100, Ctein. Congratulation (and many thanks) on reaching the milestone.

My personal memory (from c. 1982) is of the reverential hush that my colleagues and I greeted the then new IBM PC's 5 megabyte auxiliary hard drive, housed in a desktop chassis as large as the CPU unit itself. No special climate control needed!