« Photographers Can't Be Trusted? (Reuters and Raw Files) | Main | Open Mike: TOP's 10th Birthday »

Friday, 20 November 2015


Feed You can follow this conversation by subscribing to the comment feed for this post.

"...with my friend's X-Pro2"

You may want to correct that unless you're trying to push page views. ;-)

[Who said that? --Mike]

Testing the testers can be as difficult as testing the equipment.

I know a techie gal photographer who tests her tests. She is perhaps the only photographer I know with whom I avoid discussions on photography.

Just don't get carried away too much in testing equipment. You might start getting compared to the 1970's Pop Photo. Testing camera straps, lens cap holders, vinyl film can holders, etc.

I'm pretty much with you as far as testing new cameras. I sit down and figure out how to use the thing and how to set it up to do what I want it to do. Then I go shoot. I quickly learn how it performs for what I want it to do. Since digital came along I find that each new generation of camera is better at what I want than the previous one. I'm not sure what more I need to know.

Who knows how carefully mice make plans, anyway?


I have often wondered why some people have to return a half dozen copies of a lens before they find a good one, while I have always gotten a perfectly satisfactory one on my first attempt. Just lucky, I guess ...

Rabbie Burns always makes me smile. I am hoping my kilt is finished in time for Jan 25, which my friends and I will celebrate on Jan 23. Just enough time to recover for the actual day.

"The best laid schemes of Mikes and men / Often go awry."

Really? That's so bad, it's great.

Mike I happen to have the X-Pro 1 and the X-T1 and the main difference I notice in the out of camera JPG (not RAW) is that at high ISO the detail just gets smeared in the X-T1, not so the X-Pro1. Apart from that all is good.

With a camera I want to know how far I can push the ISO before the reds and blues look awful, and with a lens what the OOF highlights look like.
There are no "scientific" test for this as my awful may be your frightful. :-)

Here's an interesting approach to photography and paintings... http://photographersoilcollective.com/

See also the discussion on ownership rights to the finished product.

"I see too much when I look at pictures already." There's a whole essay in that sentence.

Also: I've always loved the sound of "Gang aft a-gley" (you have to hear it with the accent). The modern English translation loses the poetry.

I don't test lenses anymore. The last time I did, the lens in question ended up split open, filled with sand and buried in a shallow grave in the garden.

You see, I'd set up my camera on a tripod and pointed it at what I thought was a suitable combination of brickwork and vegetation to test the mettle of the gorgeously expensive Canon L glass I had recently acquired.

So curious was I about the results that, instead of dismantling the set up, I popped out the CF card and hurried inside to have a preliminary look on my computer. Upon my return, the tripod was flat on the ground, camera still attached to it. The lens had cracked open in the fall and then been buried by the culprit in a bid to hide their crime. Nearby, my then-three-year-old daughter gave an Oscar-worthy performance of pure innocence as she pottered about.

The repair bill was substantial. So, I've decided that lens testing is just too damned expensive to be worth the effort.

What Gordon Lewis wrote also reflects my perspective and experiences. If a lens or camera is a stinker or truly defective it will be soon discovered in the course of use.

As I've neither had a photo ruined by a bad lens nor have I seen a photo by others so ruined I don't have the anxiety over optical perfection that so many photographers have. The state of the art is so good, even in very modest budget optics, that I think "testing" is a waste of time. Shoot, man, shoot!

It's my experience that most lens testing is fatally flawed by initial failure to assure that the lens-to-target alignment is ABSOLUTELY parallel to the film/sensor plane.

Absolute parallel alignment of lens-to-target alignment is best assured by precise laser spot reflection, using a professional FRONT-SURFACED mirror at the target, a precision (micro-geared) mechanical stage to adjust (3 axis) target positioning, and a laser source that's pre-aligned for accurate (perpendicular) beam output.

It's quite amazing to realize how poor ordinary visual (or simple mirror reflection) alignment can be when you subsequently apply this more precise laser reflection methodology.

@ Dave Riedel - not my cup of tea, there used to be a filter in photoshop that would do what they've done to those pictures.

You see that's the problem with photography...there are far too many variables.

I've looked at camera systems for years (I was an engineer/scientist by trade), mainly out of curiosity to see if I could understand what's really going on with imaging.

I'm not sure I have "the answer" for anyone but myself, but it's this - Film (from back when dinosaurs roamed the earth), sensors, and processing are more often than not the limiting factors to image resolution.

It's perhaps hard to believe, but it's true in every single case but one (and I've looked at many hundreds of systems - http://photosketchpad.blogspot.fr/ - look along the right side of the page). Only once have I encountered an obviously bad optic (Canon 17-55 f/2.8 which normally has a decent reputation) and I returned it immediately.

Perhaps the biggest lesson for me to come from all this poking and prodding and looking at things is that commercially available imaging systems are very seldom an impediment to making a decent image. It's the brain and heart behind the eyepiece/LCD that counts.

There's testing under controlled conditions - in which you override all the defaults to equalise everything - and real world conditions where the effort of doing so is seldom considered.

The first may discover whether there is an actual mechanical issue, the second will decide whether you actually like the darned thing or not.

I'm with Gordon. The biggest bugbear I had with DSLRs was inconsistent focus. It could be very slight, but just enough to be annoying. The more consumer oriented the lens and camera were, the worse the problem was, but I have also had issues with particular lens/camera combinations on top range gear.

Change camera or lens and the issue goes away. Neither is individually broken, but a combination of manufacturing tolerances unique to that pairing just throws everything out. My D800 and 16-35 F4 was a classic case in point.

One advantage of MILC focusing is that it is self-calibrating. Using the sensor to focus is accurate every time, assuming you get a good lock. I think this was the main reason I switched. If I have a consistent focus issue I can assume that there is a fault.

Nice selfie..

Funny, because I sometimes get into testing mode myself, then later will forget whatever it is I learned. My latest was to try to learn what combination of factors gave me the sharpest photo. As in - with mirror lockup or without, using a shutter release or self timer, with or without live view, and all sorts of permutations of the above. (I think live view was the winner, but I probably need to retest to be sure)

The generally high quality of the hardware these days further complicates matters. Even the base level Canon or Nikon + kit lens (or even the right cell phone camera) is capable of very good results under the right circumstances. It's easy to come up with a test that'll distinguish between a coke bottle and a Zeiss; it's a much trickier proposition to design a test that will tease out the differences between an outstanding lens and a merely great one.

Ad-hoc testing is usually disappointing at best and completely misleading at worst.

In this case the XTRans I data stream's raw files are inherently different than the Xtrans II raw. There is a shift in the EV scale. That is, the EV ranges have different symmetries about zero EV (about a 2 stop shift)

A statistical analysis of unrendered raw data will reveal the XTrans II raw is has slightly improved signal-to-noise and hence dynamic range. This difference is almost too small to be relevant. However the extra bits in the Xtrans II's ADC are not there for purely marketing reasons. The Xtrans II analog SNR is just high enough that a 14 bit ADC can be purposeful.

Another example is the accurate, but incomplete observation that XTrans II detail is smeared (especially at high ISOs) compared to Xtrans I detail. This is so for in-camera JPEGs, but is not the case for raw rendering. The difference is in the X-T1's EXR Processor II vs the X-Pro1's EXR Processor Pro in-camera JPEG rendering. The former applies more aggressive noise filtering at all Sharpness Menu settings.

It's all relative. I have audiophile friends who have spent an absolute fortune so that they could accurately reproduce the induced distortion of 70s prog rock.

I'm with William on his observations.

In my own testing, while I find there to be a small "statistical" difference between the two Xtrans sensors, I don't really find there to be a practical one, for the most.

I do seem to observe, however anecdotally, that black and white conversions from the X-Pro1 have a bit more of that subtle Fuji magic than conversions from the X-T1.

As per the comments by several other posters, it's my understanding that there are two versions of the JPG engine. The differences you observed in the second test are well explained by the different JPG engines. Did you compare RAFs? The benefits of the higher bit depth of the X-T1 are unlikely to be seen in this shot, and so I'd expect them to be virtually identical, in whichever RAW converter you happened to choose.

Personally, I'd like to have JPG engine from my X-E1 in my X-T1.

Kodak got it, Olympus for a brief moment we're perfect and Minox came close. It's not about the lens, it's about what the customer wants and needs. Responding to this is one part of the marketing process. The lens obsession was never a criterion for me. As a business traveller, I needed a camera I could take anywhere and always use it. This was many years ago and I tried almost everything from a Rolleiflex TLR down. My need was to be able to have a camera that would fit in my briefcase when I was on a day trip to Europe or, on a longer trip but with a small bag. There were so many photographic opportunities that I could use that I wanted a camera that I could access quickly and surreptitiously and snap away. The small OM's were very good but then they produced the Oyster Shell, pocket sized Olympus - perfect, I used it for years, even snapping some great shots in the cockpit of 747's, which was welcome in those days. It went everywhere and was not an intrusion. I also had the small Minox with the folding front. Met the criteria but was not quite so snappy to use.
So, just to be contoversial I would ask if obsession with lens performance is not a bit too nerdy for most photographers - as Jim Richardson said the camera has to be suited to your needs. Given that for most of us the lens quality across almost all cameras is indiscernible, especially when we can post process it out of existence, why is it so important? Does anybody complain about lens performance of an iPhone when it is even stuck in a pocket, has fingers all around it. You can probably take a really cheap phone camera, swish its photos through almost any processor and produce as good a photo as a real camera with a real lens.
P.S. I am not a fan of phone cameras.

Since consistency and quality control on the manufacturing line is one of the main challenges in making lenses, testing one sample never tells us anything all that conclusive about a lens design. Essentially everywhere that does lens testing has this problem. I suppose it does, at least, give you a hint of what the lens can be capable of at its best. (Consumer Reports seems to be the one consumer testing group that actually understands sample variation and statistics; but what they test for in computers and cameras are more relevant to ordinary consumer use than to my own professional or serious hobbyist use.)

Weirdly i like what Fuji do with Bayer sensors

Love my X100 and use a XA1 with my other Fuji lenses.

Hope they will bring out an XT1 with a Bayer Sensor!

My attitude towards (not) testing lenses, as I wrote about back in 2010:

I Don’t Care Much about Comparing Lenses/

Surprisingly, it wasn't the death of my blog :-)

The comments to this entry are closed.



Blog powered by Typepad
Member since 06/2007