No two pieces of the same photographic equipment behave the same way. Lenses, camera bodies, memory cards, computers, printers, they all vary from unit to unit.
Some of the ways they vary are unimportant to us. Generally we don't care if the clock speed of our computer or the read/write speed of our flash RAM card is some percent higher or lower than someone else's. Sometimes it can be very important; as Lloyd Chambers has written about at length, focus accuracy in cameras is problematical at best and can vary considerably from unit to unit. Sometimes it only matters to a subset of users; most of us don't care that different units of the same camera model will have slightly different low-light noise characteristics, but if you're an available-darkness photographer who's always living on the Stygian ragged edge, that's another matter.
The problem is, any review you read is about one particular unit. You, and the reviewer, hope it's a typical unit. With time and experience, reviewers develop instincts for when something is horribly wrong; normally, though, it's impossible for us to tell if a unit is merely better or worse than average. Obviously one can't learn anything about product variability from testing one unit, which is all there is usually the time or money to do. Unfortunately, the peculiar statistics of small numbers mean that you have to test many units or you will frequently be wrong.
Let me give you a numerical example. Suppose a company introduces a new piece of photographic equipment that is so poorly made that 20% of them perform horribly. I'd certainly not want to be recommending that to my readers. How many units would I have to test to discover that? If I tested 10, there would still be a 10% chance of me not getting a single lousy unit. If I reviewed two dozen products a year and I tested 10 of everything I reviewed, two or three times a year I'd be assuring my readers that some product was well made when it wasn't.
This would not endear me to my readers or my editors, with good reason.
Not that it's feasible for reviewers to test even ten units. We just can't collect enough information on product variability to provide reliable advice. We don't even try; if we did, readers would be thoroughly ticked at having to wait an extra six months for their reviews.
There are rare exceptions. Back in the early 1980s, when I was writing for Darkroom Photography magazine, the editor Ken Werner and I were very interested in knowing what the absolute best enlarging lenses were. (The results of this investigation are in my free downloadable book, Post Exposure.) There were lots of old shutterbugs' tales out there, but precious little hard information. The magazine paid me a then-considerable sum to test every single one of the top-tier enlarging lenses made, up to and including the fabled apo El-Nikkor (the only true apochromatic enlarging lens*). I think there were 60 or 70 lenses involved in all formats from 35mm through 4x5.
I wasn't too far into my testing before I got to the Schneider Componon 50mm ƒ/2.8, a lens that was widely regarded as excellent. Well, my sample wasn't. It was so badly decentered that there was smearing of the image on-axis! At the edges of the field, around the circumference, resolution varied by considerably more than a factor of two.
Ken contacted the people at Schneider, who were appropriately appalled. A second sample of that lens behaved normally. That got me looking at quality control. I tested all the lenses for decentering. In many cases, I had the manufacturer sent a second sample; there was a lot of random sample variation. When all was said and done, I'd looked at well over 100 lenses, and maybe five were "perfectly" centered. The rest all showed a visible degree of deviation from the ideal. Nearly a third were sufficiently decentered that I wouldn't consider them acceptable.
That's when I started firmly recommending that people should never buy a lens without return privileges, and that they test it thoroughly as soon as they got it. And we're talking about enlarging lenses here, the mechanically simplest of optics. If so many of them are bad, imagine what can happen with regular camera lenses.
That's in the past. Now we come to the present. We all know that computer printers and displays vary in their color rendition from unit to unit. That's why color management and custom profiles are an essential part of any serious worker's toolkit. It's a kind of product variability that we accept as common, and that we can fix with the proper software. What, though, about the variations that we can't fix? Specifically, do two different units of the same model printer produce equally "fine-grained" prints (i.e., with uniform ink droplet patterns)? If not, are the differences between units large enough to matter?
Recently I had the reason and the opportunity to investigate that question in Epson 3880 printers, and got some surprising results. Tune in in two weeks for the full story (as TOP will be closed for maintenance next week).
[*I believe the Carl Zeiss S-Orthoplanar was also a true apochromat, although it was never readily available. —Ed.]
Ctein's weekly column on TOP appears on Wednesdays, with only occasional sample variation.
Note: Links in this post may be to our affiliates; sales through affiliate links may benefit this site. More...
Original contents copyright 2012 by Michael C. Johnston and/or the bylined author. All Rights Reserved.
Featured Comment by John Wilson: "Roger Cicala of Lens Rentals has done some interesting investigations on variability in camera equipment. He has the great advantage of having access to reasonable numbers of the same item (because he stocks them for rental) and has access to good measuring equipment. His first blog post on it is here. If you don't already read his blog I'd warmly recommend it."
Featured Comment by Allan Ostling: "Your Darkroom Photography test was a classic in its day. I pored over that article, then bought the Computar 65mm, one of the enlarging lenses to which you gave the top rating. I lost all respect for Schneider lenses, and this was reinforced when I got a de-centered Schneider zoom on a Samsung film P&S in 1996. I had a stack of those magazines from Issue #1. In 1990 I donated them to the library at the Center for Creative Photography, on the U of A campus in Tucson."
Featured Comment by James B: "There's also another variable for most modern equipment: local climatic conditions. Not extreme, but average climate from winter to summer. The manufacturers of consumer equipment all have a little paragraph tucked away somewhere in the user instructions advising of temperature limits (and it is normally pretty broad) of something like –20 to +50 Celsius (sorry my American friends, I only understand decimal). I've tested for military-grade use various thermal imaging units. These things are literally hand assembled and all are individually tested at the factory before being passed for service.
"Most are useless in a Norwegian winter or a Arabian summer. OK, those are the two extremes we test against. Most are still exhibiting significant variations in a European winter or summer. Some can exhibit variations on an April day in the U.K. in between pre-dawn and early afternoon, with a temperature difference of only about 15 degrees Celsius. Tiny variations in plastics quality and metal/plastic surfaces can twist the optics by that tiny amount as temperature changes. The principle applies just as much to commercial camera lenses as it does to military thermal imagers.
"Don't start me on humidity—that's an even worse variable.
"And worst of all is shock. You can mount an optic and calibrate it perfectly, but a big jolt and it's knocked off-axis. You can try to get it back in calibration, but what's actually the issue is that one of the 10 or so internal lenses has changed alignment, so it's a back to the factory job.
"I only have consumer lenses, less my completely beloved Nikkor 105mm ƒ/2 Defocus Control which I assume sits somewhere between consumer and pro. There's just too many variables for me to think of wasting my money on a pro lens, when I know that despite everything, so much depends on factors beyond my control."