"All Lens Tests are Wrong" from 2001. Includes a brief case against single-index ratings (apropos the discussion of the other day). Note that this is an older essay, so please don't hold me (overly) accountable if not everything jibes perfectly with what I've been saying recently; after all, we do get to change our beliefs and opinions over time, at least if we're still growing and learning...and publishing this site over the past three years has been nothing if not a learning experience for me.
Mike
Well, the success of the Lensbabies tells quite a bit about the theoretical debates on lens performance. According to conventional criteria for lens performance, Lensbabies are as bad as can be, but some pictures are nothing but remarkable. Very interesting article!
Posted by: Wolfgang Kuechle | Thursday, 12 February 2009 at 04:27 PM
I liked this the first time you published it. I like it even more now. Unfortunately, many testers have no real standard on which to base their impressions of the results. Some can't tell whether what they see is good, bad, or normal for the conditions under which the lens was being used. Others can't tell whether substandard results are caused by flaws in the lens or flaws in their test procedures. Even more unfortunate is the fact that this doesn't stop them from arguing with anyone who tries to tell them different.
Life is so much easier if you're a "shooter." As you say, if you like the results you get and you like using it, it's a "good" lens. If you don't, it isn't.
Posted by: Gordon Lewis | Thursday, 12 February 2009 at 04:38 PM
Excellent article. As much as I enjoy reading about gear, photography is about the images and the reason yours is my favorite photography blog is that you discuss images more than gear. Also the pictures most sites use to test lens quality are so BORING.
Posted by: Dave Kee | Thursday, 12 February 2009 at 07:20 PM
Mike, it's with essays like these that you show what the difference is between a mere pundit and a writer.
A pundit, like so many Internet denizens are, would have summed the issue in a cliché like "it's the photo that matters, not the lens" or something of the same ilk.
Not that the pundit is wrong. In fact I agree with him (her?). But I much prefer have a writer make a penetrating, intelligent, and extended (yet succinct) essay that covers all the major issues without cutting corners or waxing lyrical.
I think you're doing a great service to lens aficionados with a piece like that. Because many of us are genuinely interested in knowing more about our lenses, understanding how they draw the light on sensitive surfaces, but few of us actually even have half of the vocabulary needed to describe and understand it.
A patient yet stern admonition like this one is both an intelligent tribute to the passion, a respectful recognition of the authentic desire, while tempering the gearhead's hubris with the cold shower of realizing one's ignorance.
I must presume you had such a cold shower yourself at some point in your career. I hope for your own sake (and ours) that you keep having them!
Posted by: Michel Hardy-Vallée | Thursday, 12 February 2009 at 07:58 PM
Hello Mike,
Sometimes a lens test will reveal something that may not have been obvious otherwise, like for instance what photozone.de has discovered (for me at least)about the 14-45mm Panasonic lens.
http://www.photozone.de/olympus--four-thirds-lens-tests/414-panasonic_1445_3556
Posted by: Andre Moreau | Thursday, 12 February 2009 at 08:45 PM
A modest proposal
All reviewers please use the following format
column 1 = lens id
columns 2-n = score
column n+1 = commitment
the score columns (2-n) should use a simple mnemonic, eg 'omg' if barely acceptable, 'sucks' if not barely acceptable, etc.
each score column is to rate a specific use; eg landscape, studio, studio taking naughty pictures, sports....
the last column, commitment = buy, ebay, mfg*
* manufacture supplied
No misleading numbers needed!
bd
Posted by: bobdales | Thursday, 12 February 2009 at 09:32 PM
"Is flare bad? Not if you can make it expressive."
I beg you to reconsider on grounds of inanity. Is vignetting bad? Not if you can make it expressive. Is noise bad? Not if you can make it expressive. Is moire bad? Not if you can make it expressive. Is pigeon poop on the lens bad? Not if you can make it expressive.
An unnoticeable veiling flare has sucked the life out of 1000x more otherwise excellent photographs than an expressive flare has ever done any good. Flare sucks. End of story. (In my humble opinion of course, sahib.)
JC
Posted by: John Camp | Thursday, 12 February 2009 at 11:32 PM
Just curious: by 2004's Sunday Morning Photographer column, "Parameters of Lens Specification and the Properties of Lenses" http://photo.net/columns/mjohnston/column44/index.html the "I" who witnessed the Leica barbed wire demo had become "a friend". Were identities changed to protect someone?
Posted by: robert e | Friday, 13 February 2009 at 12:51 AM
robert e,
Jeez, I gotta be careful with people with memories like yours! It's like corresponding with Oren, who has a much better memory for stuff I've written than I do.
I believe that the "Leica rep story" was one that I "heard," years and years ago--might have been before 1980--but visualized so vividly that I've transformed it into "my" memory. I did go to my first camera show around that time, and it might have been one of those things I didn't literally witness but that I was told by an actual witness had happened nearby and at that time.
I'm afraid I don't remember the truth. I remember stories, and this has just entered the vast archive of "stories I know." Hope this doesn't make me sound too unreliable...
BTW there's a very interesting article by John McPhee in the current New Yorker magazine about fact checking. This would be one of those facts that would be "on author" to a good fact checker. Sadly, author is not sure. Wouldn't make the book at the NY'er.
Mike
P.S. I went back and changed it in the text.
Posted by: Mike Johnston | Friday, 13 February 2009 at 04:21 AM
Thanks for the timely post Mike! I was just wondering whether I should like my Nikon 18-200 VR or not (after shooting 10k images with it).
Posted by: Manish | Friday, 13 February 2009 at 05:32 AM
How about the Nikon 28mm f/1.4? Forget about its performance, that thing would make my camera look AWESOME.
Posted by: Jed | Friday, 13 February 2009 at 09:45 AM
Mike, re: robert e.
The older I get, the better I remember things that never happened...
Years ago, I realized that I could make a leicaphiles head spin nearly 360 degrees by telling them that my 35mm Summilux was a real piece of s**t, but it took pretty nice pictures anyway.
Posted by: Bill Bresler | Friday, 13 February 2009 at 09:56 AM
"Sadly, author is not sure."
Not the intrigue I was hoping for, but understandable and forgivable; and fascinating in another way.
In fact, while I do remember many things you wrote from when I adopted SMP as my photography primer, this is one of those stories that is so vivid and illustrative that it has a life of it's own, bending history around it.
"Were it written with a needle in the corner of an eye...", etc.
Not only was I sure that I'd been told this story by a pro photographer (in addition to reading it in SMP), but the reason I googled it in the first place was that I "remembered" the story I read in SMP being about a Nikon rep! I wouldn't be at all surprised if it turned out that the photographer who told me he saw this demo really only read your version in SMP and "remembered" seeing it for himself. At least no less surprised than if it turned out that you'd both been in the same audience. Or if it turned out that the barbed wire had transformed through retelling from a set of car keys.
In fact I have a crazy notion that that very Leica rep could be reading this exchange.
Posted by: robert e | Friday, 13 February 2009 at 02:00 PM