I like empirical tests. Especially when people get carried away and start telling me magic-bullet stories about lenses.
I haven't told this first story in a number of years, and newer readers might not have heard it, so I thought I'd retell it this morning. Many years ago I tried to empirically test the claims that there was a "Leica lens look." I made ten very good 11x14 prints, and asked Leica fans to identify which pictures were made with Leica lenses. I even told them how many of the ten pictures were made with which brands of lens—two Olympus, two Zeiss, three Leica, one Canon, etc. The handful of photographers who actually accepted the challenge did slightly less well than chance in their guesses. (The picture most often chosen as having been made with a Leica lens was made with a Zeiss Contax 100mm ƒ/3.5.)
I didn't reach statistical significance with the experiment, though, because so many Leicaphiles balked at giving it a go and guessing. Some insisted on being given further clues; some made lists of reasons why the test was not valid; some accused me of "subconsciously" taking more care in making the non-Leica prints; and some said bitterly that I had used the wrong Leica lenses. Some just refused.
One guy told me he would gladly take the challenge, but thought better of it a few days later and insisted that I repeat the comparison with Kodachrome and with the cameras tripod-mounted—because that was how he practiced his own photography. If I would only do that, he insisted, then he would be able to recognize the Leica lens shots easily, no question in his mind. I had had enough at that point, so I gave up.
My friend Jim Sherwood did an empirical test years ago. He used a Pentax 6x7 at the time—early '90s—but he had a Leica and some of Leica's best lenses, and was wondering if he could switch to a very high-quality 35mm and lighten his load in the field. So what he did was to shoot the same scenes with the Leica and the Pentax. He optimized the Leica shots by using a tripod and what was then Kodak's best color negative film, Ektar 100. For the Pentax shots he used his everyday technique, handholding with VPS (Vericolor 100, a low-contrast "portrait" film favored by art photographers at the time). What he found was that it was enlargement dependent—the Leica prints looked better at small sizes, and held their own up to about 8x10 inches; the Pentax began to pull ahead at 11x14, and the Pentax prints blew the Leica shots away at 16x20 and 20x24. Proves nothing globally, but it was useful information for Jim at the time, because he printed at 16x20 and 20x24.
An author named Stephen Benskin wrote some wonderful articles for me at Photo Techniques magazine years ago about the pioneering work of C.E.K. Mees and Mees' assistant Loyd Jones at Kodak in the 1930s. Mees and Jones essentially set the standards and methods of how B&W photography was practiced for the next 70 years, and they did it in part by carrying out empirical tests—they showed prints to large numbers of people and just asked people what they liked best. That was how Kodak settled on the design characteristics of the famous Commercial Ektar view camera lenses, too.
Kodak Commercial Ektar 12-inch lens currently for sale on eBay
The most recent empirical lens test I did was when I was over the moon about the Fuji 35mm ƒ/1.4. Beautiful, classic lens that I was sure had "that certain something"—an ineffable magic in its images. It does have the look of a classic Planar-type, a category of lens I've been very familiar with for many years. But my balloon burst when I made direct empirical comparisons with the Fuji 35mm ƒ/2 at ƒ/2 and stopped down. In direct visual comparisons, the slower lens turned out to be slightly sharper, somewhat better in the corners wide open, and really not much different at all to the faster lens in other visual characteristics. I ended up pretty sure I wouldn't be able to tell one from the other in a blind test.
So much for ineffable magic.
About the only thing you can say about Leica over the decades is that it doesn't make low-end lenses. There are no cheap consumer lenses made with downward price-pressure as the overriding concern. (Leica does cuts costs, however, at times, like everybody else.) Don't get me wrong—it's a huge advantage to be able to charge more than anyone else for everything you make. Most lensmakers make lenses that are adequate, lenses that are average, lenses that are good, and lenses that are better than good. Leica makes few lenses that are average and almost none that are merely adequate (cheap above all). They also make some wondrous lenses that Leicaphiles love...which is true of many other lensmakers as well. That's the most you can say, unless you're a fan of the brand, in which case (by definition) you're not objective. All the major makers make some great lenses. Even once-modest Sigma has made some superlative lenses because it found the rationale and the corporate will to do so.
Things have gotten very unbalanced in recent years—tests and claims have gotten way out ahead of empirical trials, and a great hullaballoo is made over vanishingly small differences. It's as though comparison shopping has become a mass neurosis. Is that because many people use shopping as an entertainment, divorced from actual purchasing? They really should get back to simply looking at pictures one of these years...and I don't mean looking at pictures for their technical properties.
As for lenses, empirically speaking, there is adequate and there is average and there is good and there is better. But there is no magic bullet in lensmaking except budget.
Mike
(Photo used with permission)
Original contents copyright 2019 by Michael C. Johnston and/or the bylined author. All Rights Reserved. Links in this post may be to our affiliates; sales through affiliate links may benefit this site.
Please help support The Online Photographer through Patreon
(To see all the comments, click on the "Comments" link below.)
Featured Comments from:
Mark Jennings: "Re 'shopping as entertainment': In some of my other pursuits I’ve seen love of gear overshadow love of the activity, its subject, and its setting. Most people spend a lot of time indoors and looking at screens, little time in the field. So maybe there’s no surprise. It’s easy to shop, hard to make good photographs. We have a consumer culture."
Mike replies: I play a fair amount of pool now, and I just successfully resisted the temptation to buy, for my cue, one of the new carbon-fiber shafts (which are both "hot" [popular] and "cool" [fashionable] right now). I already have a very good low-deflection shaft, and I like the feel of it, and it works, and I just thought, the shaft I already have is a lot better than I am. (It helps that those new ones are expensive.)
In a similar way, work cures GAS. Anytime you get to jonesing for a different camera or a new lens or whatever, just find a project and get to work on it. The GAS will go away. It works. It's the cure.
Of course, I can tell you all the advantages of the CF shafts and the plusses and minuses of every brand, so I guess your point holds. :-)
Al C.: "Talk about ineffable magic: nothing beats 'audiophile' cables. Of course, likewise, few empirical tests have ever been done, and no magical cable peddler has ever accepted a double blind A/B challenge. At least Leica lenses are of impeccable construction and a joy to use. An intrepid guy once took the jacket off an insanely expensive speaker cable, only to find a few Home Depot stock Belden wires twisted together inside.
Mike replies: Cables make a difference. But, assuming they work properly, they're just filters. Change the filter, the sound is different. Trouble is, the filter affects every system in every house differently, and there's no across-the-board qualitative change you can rely on...the sound might be different with a different filter, but only you can say if it's "better." And then it might not be better or effect the same change in a different system or with different AC.
The price of cables is merely down to peoples' perceptions of proportion and appropriateness. It just doesn't seem reasonable to spend $5,000 on an amp and $10,000 on a pair of speakers and connect them with lamp cord from the hardware store. And you can make speaker cables that sound much better than lamp cord (solid copper sounds better, more copper sounds better, purer copper sounds better, better shielding might sound better, etc.). So people think it's a reasonable protection of their other sunk costs to spend hundreds or even thousands of dollars on connecting the amp to the speakers. You wouldn't put bargain tires on a Porsche 911.
Meanwhile, from the manufacturing side, cable has a very high profit margin in an industry with very low profit margins. So there's a lot of incentive to charge more and to work harder convincing buyers with marketing.
The trouble comes in when people start fussing with different high-cost cables. Different filters, slightly different sound. No real qualitative indicators of what is "best," so they fall back on the consumer's favorite reassurance: whatever is more expensive must be better. Combine that with insecurity, gullibility, self indulgence, perfectionism, high standards and low sensitivity to price (some people just don't care about dropping five grand) and there you go.
But they're just filters. My $0.02.
Jim Arthur: "Not seeing magic lenses and dismissing 99% of the pictures we see just means we are older, wiser, and more experienced. I don’t think I can put it any nicer than that. I would normally just say I’m a round-bellied gray-beard. For me, a magic lens is one I’ve done good work with and therefore have a fond feeling for.
"Last night I dreamt I lost my EF 135mm ƒ/2 and I was not happy about it. I don’t believe it woke me but I still recall the dream this morning as I sit here eating my sweat-shop eggs. I don’t normally remember my dreams. I’m sure there are better 135mm’s out there and I’ve never done any empirical tests on it but this is my 135mm and I’m happy with it. I love some of the work I’ve done with it and it was the first expensive prime I ever bought. It may be old and unstabilized but it’s a keeper…and it’s magic to me."
Graham Byrnes: "And I once failed to distinguish red wine from white while blindfolded, as some of my friends had predicted I would. I have since avoided tests of this sort to avoid the embarrassment. If you want to get into violent reactions though, take someone who claims to be electrosensitive and put them in a room with a radio transmitter. Ask them to tell you if it is plugged in or not. So far, no one does better than chance."
John Krumm: "I'm a little bummed out about the way lens development is moving to faster/larger/heavier in order to make the shots look perfect at 200% on a screen when shooting 50+ megapixels. I'm hoping for some kind of reverse trend. If it takes AI to do it, or some other trick, so be it. Whatever works. I finally just ordered a printer ( Epson P800 using your link) and am very excited to get printing again after a long hiatus. Print making, even the digital process, seems to put all this arguing over details that don't matter into perspective."
Robert Roaldi: "In your second story, the Jim Sherwood test, if the Pentax 6x7 prints looked better at larger enlargements, what is it that they lose at lower enlargements? Why would Leica prints look better at small sizes?"
Mike replies: Better film and better lens? And, all else being equal, lenses with less coverage perform better. Of course, as Ctein says, "All else is never equal."
Those are just guesses. Actually an empirical test isn't designed to answer such questions. It's merely a demonstration of particular effects in particular sets of circumstances.
This is why empirical tests work so well for us as individuals, because we're not taking general knowledge and trying to adapt it to our specific circumstances; we're seeing results for ourselves with all of our own equipment and our own skills and habits, and evaluating the results with our own taste and judgement.
There are so many variables in darkroom work that most darkroom experts can point to circumstances where test results were not able to be replicated by other workers in other locations, even when the two parties put their heads together and tried to chase down the source of the discrepancies. I learned my own lesson along these lines early on when I read an article by Bob Schwalberg in Popular Photography about drying film on the reels using a Kindermann dryer built for the purpose. Bob had reported no problems, but when I tried it, my negatives were horribly curled and were very difficult to load into PrintFile sheets. I contacted Bob. Eventually we figured out that he had just used his regular film, which turned out to be Panatomic-X. And I had used my regular film, which at the time was Plus-X. Plus-X "took a set" in the Kindermann, Panatomic-X didn't. When I tried Panatomic-X, I had no problems either. Bob was penitent for not mentioning his film and for not having tried other films.
That's the problem and also the beauty of empirical tests: by definition, all the variables are your own variables. So they don't necessarily tell you anything more than your own results for specifically what you tested—but on the other hand they tell you your own results for specifically what you tested. :-)
Ricardo Silva Cordeiro: "The so-called Leica glow always seemed somewhat esoteric to me. Leica images always looked neutral and well balanced, but very difficult to tell apart from other lenses, nothing like the 'Zeiss look' which tends to have that in-your-face 'punch' and (micro)contrast that can make them very identifiable.
"Your comments about the two 35mm Fujinons made me think on how my mind may be clouded by some 'magical thinking.' I may have convinced myself that the ƒ/1.4 had a softer and more neutral look and the ƒ/2 a Zeiss-like rendering. This maybe to justify to myself that I need both, better not make a comparison since I really want to keep the two of them.
"Speaking of Fujinons, have you compared the two 23mm's? I’ve read some technical comparisons around, but would love to read your empirical take on this since it’s your favorite focal length and you write about lenses in different way (which I really enjoy reading). I bet that the comparison should attract some extra traffic to the website too. :-) "
Abraham Latchin: "I don't know the answer, but what I do know is a well-developed eye can become more sensitive to many things. My sister is a qualified Interior Designer, worked as Interior Director at Wallpaper magazine before moving into set design for the advertising world (starting a family was the catalyst for the change). Her eye for spaces and things is so refined she easily sees things that many would miss. The reason I tell this story is that we assume the average punter can see what a well developed eye can see, this is probably not true. There are certainly lenses with that magic. One I can see is the Olympus 25mm ƒ/1.2. It draws a picture very differently to the 17mm 1.2 and the 45mm ƒ/1.2. While the other two are lovely lenses, there is something missing. You can't always see it, but when you do, it is most certainly there. :-) "
Trecento (partial comment): "Humans are pretty good at picking out a faint signal from a sea of noise. There are some neat experiments, where participants were made to play card games using cards of two different colors. The game was fixed so that one color of cards was mildly more likely to produce a win. It didn't take too many trials before the players began to favor cards with those hands."
[Trecento's whole comment is very interesting, but long. To see the full text of any "Featured" comment, go to the Comments Section by clicking on it at the bottom of the post footer. —Ed.]
Eamon Hickey: "Re 'But there is no magic bullet in lensmaking except budget': I wonder how many millions of goofy discussions about lenses could be avoided if this basic core truth were more widely understood.
"I learned it in spades while working for Nikon, which had whole divisions making industrial lenses that were, in a strictly technical sense, far higher performers than any lenses for general photography that Nikon, or anyone else, were selling.
"The difference was that the customers for those lenses would pay far higher prices to get what they needed—millions of dollars per lens, in some cases. Now that's carrying the point to an extreme, obviously, but the principle holds.
"It can be very instructive to noodle around on the websites of the big precision optical companies to see what's going on in lensmaking beyond consumer cameras, in fields where the buyer's budgets are much bigger than the contents of your average photo dawg's piggy bank.
"One kind of surprising thing, for example, is the existence of a large, extremely high precision lens and optics (especially laser optics) industry in the United States, which has been around for many decades. But the companies don't sell consumer lenses; they are mostly defense and aerospace contractors, or industrial and research suppliers. We don't know their names, but they make fabulous lenses—far 'better' technically (I'm oversimplifying) than our camera lenses from Leica, Zeiss, Canon, Olympus et. al.—for customers who can pay hundreds of thousands or millions of dollars for them. Budget, budget, budget. (Just to reiterate, Zeiss, Canon, Nikon, Olympus et. al. also make industrial superlenses for mega-dollar customers; not dissing them at all.)
"In some ways the real magic of a lens manufacturer like Canon (or name your favorite camera company) is their ability to make what are in fact quite good lenses in huge quantities at crazily low prices.
"A company in Colorado called Ball Aerospace just manufactured the largest refractive lens ever made, which a couple of camera news sites wrote about. It's an unbelievably precise bit of lensmaking. (Ball makes the lenses in spy satellites, and they also made the corrector for the Hubble Telescope, among many other ultra high-precision optics.)
"But if you asked Ball Aerospace to make 75,000 really rather good 50mm ƒ/1.8 autofocus lenses in a year and sell them for $125 apiece, their heads would explode. They can't make a cup of coffee for $125.
"The ability to do that really is a different kind of magic, one that I don't think we appreciate as much as we probably should.
"Here's the story on Ball's new lens for a $168 million telescope, and here's a fun short read on the $50 million Hubble corrector optic and the optical genius who conceived it."
Mike replies: Absolutely. When my Dad was Director of Space Applications for NASA in the 1970s, satellites fell under his purview, and on a visit to his office once (I was a kid more or less) I got a tour of some satellite tech from one of the experts. I was shown a satallite surveillance lens and had it explained to me, and I rather innocently asked what it cost. The engineer said, "That one set the taxpayers back about two hundred and forty thousand dollars." And that was a lot of money for a lens back then!
Your comment about that cup of coffee made me laugh. Thanks, Eamon.
Christopher Mark Perez (partial comment): "'Back in the day' I worked as a black-and-white print tech for Sammy's lab on Sunset Boulevard. The other techs and I agreed when inspecting prints that we could quickly guess which format gear was used, but, with extremely rare exceptions, not the lens."
K: "While I agree with the sentiment and conclusions expressed in this article, I think some more clarity is required. While there is no magic in lens design and in the end, technology and cost will determine most of the outcome, there are differences between philosophies of lens design and manufacturing.
"I'll use examples from the lens makers that I'm familiar with: Tokina lenses are always very sharp and well built, but they compromise heavily on CA. Olympus Micro 4/3 lenses are super-sharp and contrasty, but not only is that heavy contrast an impediment, but they compromise with heavy distortion in wide angles (no Micro 4/3 lens made by Olympus or Panasonic below 40mm-equivalent has less than 3.5% distortion; many have up to 8% distortion). Fuji compromises size to some degree, but their high-end primes have just the right amount of everything, and have no distortion whatsoever. But they are master of none. Same with Pentax.
"Depending on ones tastes and goals, these different philosophies can occasionally produce distinctly different results based on the subjects."
David Dyer-Bennet: "I'm not really in doubt that the famous instances of wine experts being fooled are real. But I'm not at all convinced that that means no wine tasters are tasting accurately, or that expensive wines are just like other wines.
"Back in the late '80s / early '90s, some friends and I ran a series of wine tastings. None of us were anywhere near real experts, and the other participants graded down to people who didn't think they much liked wine. (We were wine enthusiasts, and were hoping to raise people's awareness by letting them taste a variety of things in reasonably friendly conditions.)
"We presented the wines nearly double-blind (one person did put the bottles in the bags and label them, and was in the room when people were tasting, but they avoided doing anything except tasting themselves until the reveal). We encouraged people to take notes, and we had a discussion comparing memories and notes before revealing what was in the bottles. And everybody could find the large differences, and people's preferences tended to cluster around the same wines (usually two or three favorites out of 16 wines in a tasting). Often people with different favorites agreed on what they tasted in the wines; they just liked different things.
"We were doing cheap-to-middling priced wines ($5 to $30 a bottle, 30 years ago); really fine wines will have more subtle differences. We'd pick a variety or region, and find ourselves 16 bottles of that type for a tasting. And none of us, likely, have the ability to be first-rate tasters.
"Influencing people by what you tell them is also different from working to let them reach their own conclusions; people who can be led astray still have taste buds!
"Also, it was a lot of fun!"
Mike replies: I have a very poor sense of taste and smell. For some reason my sense of smell seems to be returning in a small way, after being basically missing in action for a while. Recently I've been catching brief whiffs of smells again. But not for long, and nothing too strong. As far as tasting is concerned, I can taste, but not very well. I once made a chicken and rice stew for my ex-girlfriend and the recipe called for a number of exotic ingredients and spices as well as the juice from half a lemon. My ex-GF wasn't there when I cooked the dish and had not seen the recipe. She tasted it, rolled it around in her mouth for a few seconds, and said, "hmm, pretty good. The lemon juice is a nice touch." I was astonished—I couldn't taste the lemon juice for the life of me, even though I already knew it was there!
People often say "anybody can cook," at which I usually think to myself, "...not if they can't taste." For example, I've learned that spices are very nutritious because they can boost the antioxidant properties of food. But I have to spice more or less randomly...I can't really taste much of it all that well. I doubt I could identify many spices purely by taste even if I were eating them plain.
