I got a press release yesterday from DxOMark that says the new Nikon D800 is "now the best camera ever tested by DxOMark in terms of image quality."
"Incredibly," the press release goes on to say, "the Nikon D800 even surpasses the best medium-format cameras, which are priced more than 10 times higher!...'The new sensor featured in the D800 achieves the best dynamic range and the highest color sensitivity ever measured, taking the lead on the DxOMark scale with 95 points,' explained Dr. Frédéric Guichard, DxO Labs' Chief Scientific Officer. 'This camera illustrates the consistent improvement that digital camera manufacturers have been able to achieve in the last few years, mimicking Moore's law that has ruled the silicon industry for decades now.'"
It's been a long time since I paid a visit to DxOMark, but I'm going to make a small and not very venturesome bet: I'm guessing Nikon users like the site a lot more than Canon fans do.
Of the top ten cameras in overall score on DxOMark, four are Nikons (and two are Pentaxes). But Canon doesn't have a single camera in the top 10. Canon's highest placed camera comes in at #13. In the top 20, Nikon has eight cameras; Canon, just two. (Sony has five.) Nikon's little entry-level DSLR, the D5100, comes in just above the vaunted (but outgoing) Canon 5D Mark II (evidently the Mark III hasn't been tested yet).
As a side note, my most-used camera limps in at #124. (Breaks into patriotic song.)
Makes me wonder, though, why Nikon beats Canon so badly in DxOMark scores. Note that I did not put that in the form of a rhetorical question—if I were to ask, I already know the answers I'd get: Canon users will say it's because DxOMark sucks and/or has a biased testing procedure; Nikon users will say it's because Nikon's better. I still wonder what the real reason is, though. It's not like Canon can't make sensors. Some people like "the Canon Look" better.
Surprising, in any case, that the D800 is such a strong outlier. Here's the full report.
Mike
Send this post to a friend
Please help support TOP by patronizing our sponsors B&H Photo and Amazon
Note: Links in this post may be to our affiliates; sales through affiliate links may benefit this site. More...
Original contents copyright 2012 by Michael C. Johnston and/or the bylined author. All Rights Reserved.
Featured Comment by Bernard Scharp: "But when is DxO going to test some film stock? Who cares about these sensors that get replaced every two months? They still haven't tested Tri-X or Acros, and those are older than DxO!"
Ever-helpful Ctein replies: Here ya go Bernard, fresh from the labs: Tri-X: Low light performance: ISO 400. Noise: high, at all ISOs. Exposure Range: 13 stops, but with poor linearity. Color bit depth: 1. Resolution: 3500 lines. Happy, now? [vbg]
While I have no right to doubt DxOMark's ethos (I don't), their tests only tell part of the story. DxOMark only tests sensors and lenses. Sensor quality is important, of course, but a camera is more than its sensor. Take a look at Olympus and Panasonic micro 4/3 and you'll see that, despite both brands sharing the same sensor, Olympus usually has the edge when it comes to image quality. Theoretically they should have the same performance, but they don't.
Then there's the old objectivity vs. subjectivity debate, with DxO ranks and our eyes reaching different conclusions. I'm not a Canon fan writing in rage, but I believe there's a lot more than megapixels going on inside a camera. Our eyes are the ultimate testers when it comes to image quality. We should choose a camera if the images it captures are to our liking, not because its sensor is highly rated. If the Nikon D800 appears to you as having the best image quality, buy it - but do it because you have compared it to other cameras and found it to be superior, not just because it rates so highly at DxOMark's tests.
Posted by: Manuel | Wednesday, 28 March 2012 at 01:56 PM
Paging Ctein, Ctein to the white courtesy telephone ...
Posted by: Ben Rosengart | Wednesday, 28 March 2012 at 02:08 PM
I smelled the problem a couple of years ago when Canon refused to innovate in order to keep the distance it initially created vs. Nikon and others in digital quality, and instead started using bean-counters' mantra "why would we do that, when competition is still playing catch up?" (my recollection of a Chuck Westfall's interview). Well, problem with that philosophy is that competition is not only aiming to catch up, but to surpass. And they did. Translucent mirrors (Sony), high-iso quality (Nikon), dynamic range (Pentax, Sony, Nikon), megapixels, mirrorless bodies, etc.
Posted by: Slobodan Blagojevic | Wednesday, 28 March 2012 at 02:18 PM
"Of the top ten cameras in overall score on DxOMark, four are Nikons (and two are Pentaxes). But Canon doesn't have a single camera in the top 10. Canon's highest placed camera comes in at #13. In the top 20, Nikon has eight cameras; Canon, just two. (Sony has five.) Nikon's little entry-level DSLR, the D5100, comes in just above the vaunted (but outgoing) Canon 5D Mark II (evidently the Mark III hasn't been tested yet)."
Isn't it more about who made the sensor and not who made the body that the sensor is sitting in?
With the exception of Canon and Fuji, pretty much everyone else uses Sony sensors or Sony technology in their cameras ...
So, it really is Sony that rules the top 10...
Posted by: Harry Lime | Wednesday, 28 March 2012 at 02:23 PM
Slobodan,
Now that you mention it, I'm remembering that a few years back, Canon had the reputation for having the sensors with the best high-ISO performance. Wasn't that the case? I don't think I've heard that very much in recent years, but weren't there a few years in there when everybody took that for granted?
Mike
Posted by: Mike Johnston | Wednesday, 28 March 2012 at 02:25 PM
Isn't is a matter of product cycles. Two years ago Canon looked the best in the tests. Now Nikon is out with new cameras.
Posted by: ScottKemp2 | Wednesday, 28 March 2012 at 02:31 PM
I really think something is wrong with these Dxo tests, and not because I've been using Olympus cameras over the years that have held steady at 56 points or so. I don't like how little evidence they provide for their numbers. Seems like some kind of magic machine we are just supposed to believe. I know when I look at dpr studio samples of the D800 and the Canon MkII and mkIII I see more high iso noise from the D800, not less.
Posted by: John Krumm | Wednesday, 28 March 2012 at 02:46 PM
Most of those Nikon/Pentax/Sony cameras are using Sony sensors (not all the Nikon FFs though). Canon hasn't put out a new APS-C sensors in quite a while and when they do they're at a slight disadvantage due to the slightly smaller size. It's also useful to note that dxomark does not test resolution, and while their scores show inferior results for the 7D/60D, other tests show that they hold detail very well up into high ISOs. That said, there was a time when Canon sensors were superior and Sony's were "noise generators" (if you listen to the hyperbole) and the tables do seem to have turned.
Posted by: Dennis | Wednesday, 28 March 2012 at 03:08 PM
Canon's response to the rumoured D800 from Nikon was "we could if we want to'
Shouldn't that have been - 'if our customers want/need it then we will deliver it'?
George
Posted by: George Barr | Wednesday, 28 March 2012 at 03:08 PM
Looking at the site, it seems like the big difference in the components of the D800 and 5DII scores is in the "low-light ISO" category.
Before you wrote this article, you should have actually read up on how the scores are calculated: http://www.dxomark.com/index.php/About/Sensor-scores
The total score is based on an average of:
(1) color depth performance - number of bits (the number of colors that the sensor is able to distinguish, measured in bits)
(2) maximum dynamic range performance - exposure value (greatest possible amplitude between light and dark details a given sensor can record)
(3) low-light sensitivity performance - ISO sensitivity (highest ISO setting for a camera that allows it to achieve an SNR of 30dB while keeping a good dynamic range of 9 EVs and a color depth of 18bits.)
Posted by: Diego | Wednesday, 28 March 2012 at 03:31 PM
From DXO's preview of Canon's 5D3:
"Our DxOMark comparative tests for these two reflex cameras and their sensors will definitely measure and determine the veracity of these two principal claims:
The gain in resolution that the Nikon D800’s 36-megapixel sensor should provide.
The gain in sensitivity for the Canon EOS 5D Mark III (we’re betting on this)."
It seems to me that they are hinting that the D800 might not be #1 for long, or that at the very least, Canon's standings will improve.
Posted by: Gary S | Wednesday, 28 March 2012 at 03:49 PM
Maybe DxO Mark wanted to release a "Nikon tops our charts" statement before the new Canons overthrow Nikon again, so that they can announce the new Kings and Queens?
That said, I think that Nikon simply got a) a big hit (the D700/D3/D3s sensors) and b) lucky - that sony produced this excellent sensor in the D7000/D5100 (and K5, A550,...).
Besides, I think that DxOMark emphasizes the circumstances under which one get's very good photographs (i.e. high DR, high SNR, etc.) but not so much where you get useable pictures (which is interesting for photojournalism where often web and small print sizes are relevant).
Posted by: mike | Wednesday, 28 March 2012 at 03:49 PM
DxO is great for photographers who buy the "best" cameras based on numbers. But for photographers who photograph actual humans, Canon offers better color. A lot of photographers know that based on their own experience, despite what Dr. Chief Scientific Officer says.
Posted by: Zlatko Batistich | Wednesday, 28 March 2012 at 03:53 PM
... and does it matter that much, I wonder? Yes, I am a Canon user, but I am yet to feel the need to upgrade my 1100D. I do think it's about the brain-eye-lens triad, before the sensor or processor.
Posted by: Carlos Ferreira | Wednesday, 28 March 2012 at 03:55 PM
The big advantage for the Sony-sensored clan is in read noise at low ISO. Sony's scheme for reducing read noise gave them a big bump in the "landscape" (dynamic range) part of DxO's tests. So far, Canon simply hasn't been able to match that technology with anything that worked as well.
Posted by: david | Wednesday, 28 March 2012 at 04:15 PM
You're asking the wrong question, Mike. The right question is why are Sony's sensors better than Canon's?
The answer might surprise you: Sony discovered a mine of fairy dust a few years ago and are keeping it all to themselves.
There is another far-fetched theory saying that a flying saucer crashed into one of their factories and they've been back-engineering its imaging technology, but that's just silly.
As an aside, I'm glad Canon isn't producing 36MP sensors. 59% of Nikon shooters agree with me on that, by the way.
Posted by: Miserere | Wednesday, 28 March 2012 at 04:17 PM
I really don't know what to make of these tests. Grain of salt, I guess, just like everything else.
Honestly, there's so much noise on the internet about every camera that the only way to wrap your head around one is to click the "add to cart" button and fire off a couple hundred frames for yourself. (And thanks to excellent return policies on several sites this is a fairly low risk proposition.)
Posted by: Burt | Wednesday, 28 March 2012 at 04:22 PM
DXOmark's D800 measurement has been corroborated by independent measurements by other people. It is a more or less exact up-scale of the D7000 Sony sensor, so the results are utterly predictable.
The only question is how much you care about the ratings and conclusions.
As with any objective measurement, it's not everything. Photography is a subjective medium. A car may pull more g in a corner (measurable) and accelerate to 60 faster (measurable) and even do better lap times repeatedly (measurable) and still be awful to drive on a real road because the suspension is rock hard and the steering kicks back on bumps.
I switched from Canon to Nikon, then Minolta and Pentax before going back to Nikon. From a handling perspective I would take the Pentax K series over all comers, but the D700 just works better (AF, exposure, DR, ISO). However it is a great deal less pleasant and more fiddly to use.
I will buy a D800 at some point, despite the price hike. I am wary about the impact to my hard drive and the amount of portable storage I am going to need, but I have come to rely heavily on the robust AF on their FF cameras and the fact that you can push the files around mercilessly in PP without messing them up.
But I still miss the handling of Pentax cameras and the default IQ when they worked was lovely with almost no PP effort at all.
So objectivity is all very well, but DXOMark certainly does not tell you all you need to know about a camera and nor does it make such a claim. But if you understand it's limitations it can still be very very useful, just as lap times tell you everything you need to know...if you plan to go racing.
Posted by: Steve Jacob | Wednesday, 28 March 2012 at 04:26 PM
The new Nikon's are using Sony sensors so I'm more surprised that Sony cameras aren't listed at the top at the same rating as the Nikon's. Surely Sony wouldn't make a better sensor, let their competitors use it and not themselves.
I guess the real comfort here is that the folks at Nikon have a better understanding of the technology they use and create and know how to maximise performance. The score though really seems to aimed at finding the best all-rounder, Canons beats Nikon on some aspects, Nikon beats Canon on others. So a smart shopper will take a closer look and determine which camera suits the aspects most important to them.
My DSLR is at #65 but 90% of my shooting is with my compact at #153 which I'm very happy with.
Posted by: steve | Wednesday, 28 March 2012 at 04:41 PM
Could it be that the DxO score is not that relevant to the overall performance of the system (system being sensor, processor, lens). Not to critique DxO since they do waht they do best and that is to stick to a meticulous testing protocol.
But my ten years experience in testing makes me aware of the so called "protocol blindness". When things have changed old protocols can become insignificant because new technology compensates. So a sensor score of 52 can in theory be compensated by better noise cancelling algorithms etc. A dynamic range of 14 EV in a sensor on the other hand can be ruined by a lousy lightmeter......that sort of thing. So testing the sensor when everything surrounding the sensor is in constant flux, could pose the real danger of "protocol blindness" which creates testresults that are compareble with older results but are also very far removed from the actual day to day reality in using the product.
Greetings, Ed
Posted by: Ed | Wednesday, 28 March 2012 at 04:43 PM
Whilst I don't doubt DXOMark's testing methodology, the scoring part of it for me is where the devil in the details lie. The DXOMark is a score (out of 100) that I'm assuming is mathematically calculated. If it is out of 100, then it is clearly non-linear, and it is the result of combining the scores of Resolution, DR, Color Depth, and Low Light Performance.
Which of course means that the score doesn't mean much to you if what YOU want is weighted / calculated differently. Let's say low light performance is what you need. Then the algorithm that DXO Mark uses to calculate this is clearly not going favour that for you. I'd say the same for each manufacturer -- each camera is targeting a particular customer type, so it will naturally favour different characteristics.
On saying that, it was after reading various user reviews and handling a camera physically in a shop, that I was still unsure on a purchase -- and DXO Mark pushed me over. I'm looking at you right now K-5. I'm not at all surprised the D800 came out on top, having seen the sample photos from that sensor.
Pak
Posted by: Pak-Ming Wan | Wednesday, 28 March 2012 at 04:45 PM
"he new Nikon's are using Sony sensors so I'm more surprised that Sony cameras aren't listed at the top at the same rating as the Nikon's. Surely Sony wouldn't make a better sensor, let their competitors use it and not themselves."
Steve,
From all appearances (i.e., this is not some sort of inside information), Sony isn't going to continue to participate in producing full-frame cameras. So it won't be using the FF sensors it provides to Nikon.
Mike
P.S. What's with everybody misusing apostrophes today? Seems worse than usual.
Posted by: Mike Johnston | Wednesday, 28 March 2012 at 04:45 PM
Dear Diego,
Why do you assume Mike hasn't read all that stuff? He's just reporting a particular result. He hasn't said anything like, “This proves Nikon has the best camera in the world.” He is the reporter, not the creator.
In truth, Mike and every other savvy reader understand exactly what the single number rating is about. It's for people who don't want to read the detailed results or don't care or can't understand them. It is DxOMark's estimate of how well a camera would satisfy 1000 randomly-chosen serious photographers. For any particular photographer, personal needs will be different and they'd be evaluating the details.
pax \ Ctein
[ Please excuse any word-salad. MacSpeech in training! ]
======================================
-- Ctein's Online Gallery http://ctein.com
-- Digital Restorations http://photo-repair.com
======================================
Posted by: ctein | Wednesday, 28 March 2012 at 05:03 PM
Didn't scientists discover that some particles travel faster than light. That is until they found out that their instruments were wrong.
Canon user
Posted by: Rex Kersley | Wednesday, 28 March 2012 at 05:29 PM
Where did you get the info that Sony wont be making FF? I believe they have indicated that a ff 36mp is coming this year. It will use the Sony slt technology. I'd bet that Nikon is using the same sensor.
Posted by: Mike Fewstert | Wednesday, 28 March 2012 at 05:32 PM
Does DxOMark pronounce their name "dee ex oh" or "dee times oh?" Either way, I don't care any more than I care about their tests. It's got nothing to do with photography. And I use Nikons.
Posted by: Jon Porter | Wednesday, 28 March 2012 at 05:39 PM
Seems to me that Canon was generally ahead in DSLRs until the D3 hit. Since then, not so much (though they held the high-res crown for a while).
I'm actually rather surprised that the new Nikons blow away the new Canon's; I thought it was clearly time for the next leapfrog. But apparently not.
Posted by: David Dyer-Bennet | Wednesday, 28 March 2012 at 05:47 PM
As someone who has extensively used cameras at different ends of the DXOMark spectrum over the last few years, I can vouch that their numbers do have a direct correlation to real life experience in many ways. The dynamic range and bit depth numbers appear to be right on. My D7000 has much better dynamic range than my 5D MkII did and both are miles ahead of the 12 megapixel sensor Olympus has ran into the ground. To me, these numbers are crucial in real world scenarios and allow the camera to perform at its best in as much situations as possible.
The DXOmark numbers do not reflect resolution in any way, from what I can tell. A full frame camera will (generally) provide crisper results over a crop camera. If you look at DXO's lens ratings, this becomes quite apparent. A d7000 with a good zoom is rated around a 12-14. A 5D Mkii with a good zoom is at least a 19-20. A 5d MkII with a prime lens is far and away better than a crop format camera with the same lens.
Canon had the lead and lost it. It appears that they've hit a wall as far as sensor development goes. They've been using the same APS-C sensor for quite a few years and it trailed behind the Nikon D90 and D300 from the get-go.
Posted by: Josef | Wednesday, 28 March 2012 at 06:07 PM
If their current camera scores at 95, what will happen when Nikon introduces a 10-15% better camera two or three years from now? Will the DxO scale have to go up to 110?
Posted by: MM | Wednesday, 28 March 2012 at 06:10 PM
This may be considered off topic and for that I apologize. However I had a thought that what if sensors are the new film? (in a way they already are - but I mean in the artistic sense)
what I mean by that is we used to buy velvia or ektachrome, for very distinct reasons. Is it possible we should maybe start thinking of sensors more in that vein than than just in the megapixel/noise/low light concerns?
Anyway it struck me as something to share.
Posted by: Matthew Langley | Wednesday, 28 March 2012 at 06:23 PM
DXO results track fairly closely with what I've found in use. The nice thing is that while they give you the option to compare based on per-pixel performance, they also provide per-image performance.
One of the things I can't stand about many sites (e.g. DPR) is that they compare images only at the pixel level, ignoring the advantages that extra pixels have when using the same reproduction size.
Posted by: Michael Bernstein | Wednesday, 28 March 2012 at 07:13 PM
Comparing the just-introduced Nikon D4 and D800 to the 3 year-old Canon 5DmkII is not a fair comparison. We should at least wait until the equivalent Canon 1D X and 5DmkIII are tested before drawing conclusions.
Posted by: Fazal Majid | Wednesday, 28 March 2012 at 07:14 PM
Mike, the rumor mill (and by that, I mean the reliable portions, including comments from the UK Sony spokesperson Paul Genge) all point to a FF Sony. For a while, there was a smidgeon of hope that a non-SLT was under consideration, but that hope has been pretty well dashed. The introduction of a $13000 500/4 also suggests that it's just a matter of time.
Certainly Nikon rules the top of the chart because nobody else is using either Nikons FF sensors or Sony's new 36MP sensor (the one in the D800) yet. But when it comes to APS-C, Sony's ratings are hampered by the fixed mirror in the SLT. You'll see, for instance, that the A580 and NEX-5n rate higher than the A55 which uses the same sensor. (The Pentax rates the highest; I'm not sure what tweaks each manufacturer does with these sensors). Soon, we'll likely see a Nikon D400 sitting up there with the same 24MP sensor that's in the NEX-7 (and the A77 which again suffers from the fixed mirror).
Posted by: Dennis | Wednesday, 28 March 2012 at 07:18 PM
I find it pretty easy to ignore most DXO numbers - especially the 'overall score.' I look at their dynamic range measurements, ISO ratings and noise threshold, because these make some kind of real-world sense to me.
The Nikons with Sony sensors are besting Canon because of the two-stop advantage in low-ISO dynamic range. Now, I'm not entirely sure that having over 14 stops of DR at base ISO is relevant to most people and most photography, but I'm sure it must be for some folks. (I was under the impression that your average piece of photographic paper couldn't hold anywhere close to that much range.)
If you look at the hard DXO measurements for the D700, the 5D and the 5D II they run pretty close together, to the point that their differences are probably irrelevant to most users.
If you compare them to the D800, you're getting more megapixels and base ISO DR from the D800 but as the ISO starts to climb the difference narrows a lot.
For someone who shoots at 1600-6400 much of the time, it looks to me like one's upgrade money is best invested in a 24/1.4 or 35/1.4.
Posted by: Matt P | Wednesday, 28 March 2012 at 07:50 PM
Sometimes DxO is okay to make sure that the sensor of your next camera is okay.
Technically by pixel Canon is somehow out of this race. Compared to Phase One. Again technically by pixel D800 Sensor might be better but as a whole Medium Format rules.
One important thing for the 135 Coneheads is that Medium Format by physics (and optics) is a different ballgame than 135 format. Don't talk that 135 overtakes MF. It makes you look stupid. :)
Posted by: `/1nc3nt | Wednesday, 28 March 2012 at 07:55 PM
This reminds me of all the "bench tests" of lens sharpness. They compare degrees of resolution or whatever under conditions that almost no photography takes place and can't be seen by the naked eye in the results of everyday photographs. I found the "Comparator" link the other day or the results on DP Review of much more practical use to most buyers but even those are approximations. Numbers games don't mean much to me.
Posted by: Jim Bullard | Wednesday, 28 March 2012 at 08:05 PM
Isn't the most likely explanation is that the Nikon managers and engineers are determined to do as well on the DxOMark as possible, so they do? It's like any other area where a single salable number is generally paid attention to (the processor speed for PCs a few years ago, the SAT score for college admissions). Whether the number really measures performance, some people will put an inordinate emphasis on it. Others will pursue a more holistic approach and perform just as well, if not better.
Posted by: JPC | Wednesday, 28 March 2012 at 08:05 PM
I suppose Canon just point to the comparative sales charts, and laugh.
Posted by: Michael W | Wednesday, 28 March 2012 at 08:31 PM
Trying to quantify something that is more qualitative (in this case, image aesthetics) is always controversial.
Personally I don't find these kinds of reductionist rating scores to have any practical value.
Posted by: Kelvin | Wednesday, 28 March 2012 at 08:38 PM
The DXOMark is a score (out of 100) that I'm assuming is mathematically calculated. If it is out of 100, then it is clearly non-linear, and it is the result of combining the scores of Resolution, DR, Color Depth, and Low Light Performance
The composite score is open-ended, not out of 100. To first approximation, it's a linear combination of the individual scores and the logarithms of the individual scores. Obviously the individual scores are more useful than a single weighted average.
Posted by: david | Wednesday, 28 March 2012 at 08:50 PM
"now the best camera ever tested by DxOMark in terms of image quality."
... in terms of image quality as defined by DxOMark ... which may or may not be of interest to people who buy cameras and take pictures.
Remember the megapixel race?
Posted by: Speed | Wednesday, 28 March 2012 at 08:59 PM
When Sony went to on-chip AD conversion a few years back with the EXMOR line of sensors (in most Nikon and Sony DSLRs,) they started leaving Canon behind in sensor tech. The DR advantage of the Sony sensors is tangible, and even the new 5Diii looks to be plagued with the shadow banding problems of the 5Dii.
Nikon tends to have a little better DR and noise control with Sony sensors as Sony does themselves with their own sensors, but this is partially because Sony users a rather opaque color filter array that, while providing the best metamerism, blocks a little more light from the sensor, thus gaining more noise. It's a trade off. Nikon trades color quality for noise performance. Canon does the same.
Posted by: GH | Wednesday, 28 March 2012 at 08:59 PM
Apparently, published D800 images are cleaner than 5D3s' to my eyes, and it matches to the DxO scores.
Cleaner is better?
Some Audiophiles still prefer tube equipments despite of their rather poor spec. Many (mostly amateur) photographers enjoy old soft lenses, too. So, spec is not everything, but...
I personally think the camera body should be as clean as possible, although I do not always prefer the sharpest / cleanest lenses. A camera body is nothing but an Analog to Digital converter in terms of the IQ, and it should be transparent.
Canon produces attractive pictures, no question, but I always feel some kind of artificial coloration, and I feel my photography is controlled by Canon Empire.
Posted by: dbm | Wednesday, 28 March 2012 at 09:04 PM
Dennis, many people think the sensors are not from Sony, but Renesas.
The history says, whenever Nikon used Sony sensor, Nikon always officially announce the sensor is Sony, but when they use some other brand, they always do not announce the sensor make. For D800, they did not say it's a Sony, so it's not a Sony, most probably.
Posted by: dbm | Wednesday, 28 March 2012 at 09:17 PM
Maybe the difference is that Canon is too busy focussing on video to be bothered with mere stills photographers. (I'm a long-term Canon user.)
Posted by: Martin Doonan | Wednesday, 28 March 2012 at 09:36 PM
To me, it doesn't matter very much. Yes, I shoot with a Canon, but it's an original 5D. I was not anxious to upgrade when the shiny new 5D MK II came out, nor was I tempted to sell everything and buy a Nikon D700.
Am I tempted now? Not really. If I feel at all inadequate with my current gear, I just remember what ISO 800 film looked like, and I feel much better. Could I use more dynamic range? Sure, but my 5D is somewhat like shooting Provia 100F in terms of dynamic range, (actually somewhat better, if my memory is correct) and I'm happy with that.
Posted by: Dillan K | Wednesday, 28 March 2012 at 09:51 PM
Like any other synthetic benchmark it's very good at what it does, that is at generating data for its own purposes that has little to do with the real world. Is it really surprising that a 36MP sensor is sharper than a 24MP sensor? Well, not really. All fanboys (either camp) can gloat/sulk as much as they like but it really means very little in real life applications.
Posted by: Michal Rosa | Wednesday, 28 March 2012 at 09:53 PM
And poor, old David Burnette struggles on with an antiquated original Canon 5D and somehow manages to make a living with it. Poor guy. How on earth did he ever come to be ranked as one of the top photographers around? It couldn't have anything to do with the fact that his pictures are just better, could it?
Posted by: Dave Jenkins | Wednesday, 28 March 2012 at 10:17 PM
control testing performance and robustness are two different things. this is nothing to new to anyone, and perhaps this is not what is happening: Nikon is far superior to Canon in testing performance, and real world robustness.
but can the eye tell the difference? it is the eye (and all kinds of other biases) that determine the satisfaction with a camera's imaging performance.
this harks back to the measurements of receivers with perfect 8 Ω real impedance... and speakers don't behave that well.
I am assuming that the Nikon glass is just as good as Canon's and well matched for their sensors.
Posted by: fernando | Wednesday, 28 March 2012 at 11:20 PM
"Paging Ctein, Ctein to the white courtesy telephone ..."
Hoo ha ha!
Wonderful, Ben!
Dave
Posted by: Dave Fultz | Wednesday, 28 March 2012 at 11:33 PM
If you were to create a list of say 10 photographers who have been incredibly influential over the last 20 years how many of them even know what DxOMark is let alone look at it? Trying to judge an artists tool by numbers risks being foolish.
Posted by: Threeark | Wednesday, 28 March 2012 at 11:33 PM
I just heard a rumor that DxO has changed their top scoring camera to a Monet painting.
Them's the breaks!
Posted by: Mark | Thursday, 29 March 2012 at 12:00 AM
I have found some of DxO's figures useful and they do relate to what I see in cameras I own. As mentioned by others, the dynamic range and ISO/noise figures are especially useful and can mean something for real world photography. However, as everyone already knows, the numbers alone don't tell us everything and a probably pretty far down the list of important things in photography.
Posted by: David H. | Thursday, 29 March 2012 at 01:10 AM
I see my Nikon D200 has not improved in "score" in the 5 years I have had it.
However, my own photography has slowly hauled itself out of the mass commonly known as "tourist snapshot", and so I am grateful.
To be honest, a DxOMark score is about as useful to me as a car manufacturer's published 0-60 time or fuel consumption. They do not reflect how I drive my car, but were a benchmark when I bought the car.
I do suspect that - combining camera score and car analogies - if I were to trade my D200 for a D800 - I would drive the D800 with Ferrari performance in the same way I drive my D200, like my elderly Volvo. DxOMark therefore represents to me a good reason to keep what little money I have in the bank.
Posted by: James B | Thursday, 29 March 2012 at 02:26 AM
"Canon produces attractive pictures, no question, but I always feel some kind of artificial coloration, and I feel my photography is controlled by Canon Empire."
Create a colour profile for your camera and use a RAW converter that can read the profile. Hey presto - accurate colours!
Posted by: Roland | Thursday, 29 March 2012 at 02:47 AM
I wouldn't say the Sony slts are "hampered" by the fixed mirror. I'd suggest there are more plusses than negatives to SLT technology.The light loss is very small and adequately compensated for by the sensor performance.
Posted by: Mike Fewstert | Thursday, 29 March 2012 at 03:10 AM
You think canon partisans are unhappy, consider the poor leica buyer! If you believe dxomark, it is a clearly inferior sensor.
Except, it isn't. Mind, i couldn't care less either way, and own three brands of digital camera (m9 being my primary). I tested my 5d2 pretty extensively against the m9, and there is no way that the m9 is lower dynamic range than the canon; quite the opposite. In fact, i just was shooting side-by-side comparisons between my m9 and the new d4; i was frankly surprised to find that the leica files offered recoverable shadow details which the nikon couldn't match.
Now, i expect that the d800 will surpass the leica sensor in every way; they seem to have hit the ball out of the park with this one (and it wont affect my use of the m9 one way or the other). But, it remains troubling that the dr score, which does seem to track real world differences between, say, the 5d2, the d7000, and the d4 quite well, seems downright misleading in the case of the m9 (or for that matter medium format sensors). Maybe they have an issue with the way their instruments interpret ccd (as opposed to cmos) sensor performance, compared to practical perception; maybe it is something else. All i know for sure--having hung meter wide prints from my 5d2 next to ones from the m9 in galleries, and run into cohesion problems--is that it is wrong.
It is actually quite frustrating, because the dxo scores often seem to be useful, and it would be great to have a reliable resource of this kind. But now, when the m10 is released and dxomark declares it inferior, i won't know whether to believe them or to ignore them. Yes, of course other factors than sensor performance will be primary for me anyway, but it would still be nice to know what i might be stepping into.
Posted by: S. Chris | Thursday, 29 March 2012 at 03:25 AM
Hip, hop, hip, hop.
Which frog is now on top?
Oh, look, today it's Fop.
Turn my back and now it's Bop.
Isn't it interesting that Kirk Tuck, erstwhile TOP contributor, is very happy to use m43 for his personal, arguably more artistically free, shooting. What is the difference between his mind and the mind of a frog?
Posted by: Arg | Thursday, 29 March 2012 at 05:10 AM
As many on here are saying, there are ratings and then there's "look" and reality. I was pretty happily using Canon 20D, 40D series cameras professionally and felt the need to up grade. After reading all the dope, I selected the Nikon D90 just to put my toe into the Nikon arena and decide if I wanted to buy more lenses and get more involved. The D90 chip was supposed to be better than the D300 and 300s, so it was a good way to analyze the possibilities.
Well, to Steve Jacobs point, the camera and system was far more "fiddly" and less pleasant to use. There were multiple ways to set the same things, which was confusing, the auto-focus was less than intuitive, and the manual focus was impossible to use/see. But the worst thing, is altho the chip was highly rated, I've never been able to "game" a setting that gives me what I want. It seems to have virtually no ability to get detail when you "card-fill" a subject, and I keep messing around with the contrast, saturation, and HDR to try and get it, but most often have to use a fill light, which is giving me a look I don't want.
What the Nikon chip does seem to do flawlessly, is give me that over-contrasty, over-saturated "juiced" unnatural color that all the "kid" art directors highly prized a few years ago. Someone who doesn't know all that much about professional photography and hasn't shot 30 years of transparency probably thinks it looks "great"!
Hence, I don't like this camera every time I pick it up, and shy away from using it. On the other hand, I grabbed a buddies Canon digital Rebel the other day, and without reading a thing was able to set it the way I wanted to and got some great results without even thinking about it.
I guess what I'm saying is the proof of the pudding is in the eating. DoX can say whatever they want, I'm never buying another camera without renting it and trying it!
Posted by: Tom Kwas | Thursday, 29 March 2012 at 08:22 AM
Just for fun, let's run DxO's test on 1998 Velvia and Provia and see what happens, shall we?
Why, Provia wins. It records more colors. It has a higher SN ratio. It has better high ISO performance. Thus, it gets a higher number in the DxO test for it.
Now, how many of you shot Velvia because "it was better"? ;~)
If you get mired in the tree details, you lose sight of the forest.
Posted by: Thom Hogan | Thursday, 29 March 2012 at 09:01 AM
Perspective from Erwin Puts:
"...the new generation of Nikon and Canon high-end cameras is without doubt a techno-freaks dream, but one may seriously question their ability to produce better pictures than their smaller and more agile competitors. ‘Better’ in this context is not more linepairs/mm, but more visual impact and emotional content."
Posted by: Dave Jenkins | Thursday, 29 March 2012 at 09:17 AM
"Makes me wonder, though, why Nikon beats Canon so badly in DxOMark scores."
That's the wrong question to ask, even rhetorically... A better one is "who the h--- is DxOMark and why should I bother to care about them?". What are they selling that they are motivated to anoint a new "king"?
Posted by: Derek Lyons | Thursday, 29 March 2012 at 10:01 AM
Let's take that D800, and the IQ180, and produce a 32x48 print and see which one has more resolution, dynamic range and color accuracy. Something tells me it wont be the D800.
Of course, DxO rates the APS-C Pentax K5 to be better than the Hasselblad H3DII 50, Leaf Aptus75S or the Phase One P45 Plus. Why don't we do a challenge. Let's take the H3DII 50 and the Pentax K5 and produce a series of 32x48 prints for the public to see. If the public chooses the Pentax K5 as producing the superior print, I'll buy them the K5. If the public chooses the H3D 50 as producing the superior print, DxO can buy me the H3D 50. Are they up for a challenge....I highly doubt it!
Let's stop discussing these ridiculous DxO rankings as they are completely and totally worthless with no link to reality.
Posted by: David Luttmann | Thursday, 29 March 2012 at 11:11 AM
Thanks Ctein! [vbg++]
Posted by: Bernard Scharp | Thursday, 29 March 2012 at 11:40 AM
David Luttmann,
So you're going to base your own conclusions on imaginary prints you haven't seen, based on your own native assumptions?
A strong assertion doth not a more powerful argument make. At least DxO actually runs its tests and looks at its data.
Just sayin'.
Mike
Posted by: Mike Johnston | Thursday, 29 March 2012 at 11:50 AM
"What are they selling that they are motivated to anoint a new 'king'?"
Derek,
DxO Labs is a French software company. According to Wikipedia, it sells DxO Optics Pro, a personal computer package that corrects for various optical aberrations, notably image distortion, with corrections tuned to particular lenses and cameras; DxO FilmPack, which emulates the appearance of various conventional films digitally; and DxO Analyzer, which is used to measure lens distortions. It also provides image processing software OEM to camera manufacturers.
Mike
Posted by: Mike Johnston | Thursday, 29 March 2012 at 12:04 PM
At this point, isn't it all just so much measurebation aimed at the people who spend too much time online playing Camera Top Trumps instead of using their "better than yours" cameras to take real photographs with?
It's not like the market is awash with current-model 4/3 and larger sensor cameras which underperform in image quality badly enough to be an actual problem for most people, is it?
Posted by: Paul Glover | Thursday, 29 March 2012 at 01:37 PM
Now i have a Leaf Aptus 75 which i have had for over 5 years now and it has the most wonderful colour and sharpness , and its not rated as highly as the D800 , that sounds like a tall story. not to say i would not like a Nikon D800E as well.
Posted by: Terence Hogben | Thursday, 29 March 2012 at 02:20 PM
David Luttmann,
I'd take you up on that test. Only we'll be making prints from a night football match with only sodium floodlights for lighting :)
Make it a studio shoot with complete lighting and subject control, the results change.
The scores correlate to something. Whether they correlate to the right thing for a given person's needs is another matter entirely.
Posted by: Kwasi | Thursday, 29 March 2012 at 06:02 PM
Just curious...
Isn't Nikon beating Canon rather well?
Posted by: Dave | Thursday, 29 March 2012 at 07:11 PM
Dear David L.,
Well, you just know I have to respond to your post [obsessive grin].
First start, DxO made no assertions about the resolution of the new Nikon camera. In fact they close their discussion by commenting that they will be very interested in seeing if the resolution in real life lives up to the number of pixels. So you've got no bone to pick with them there.
Regarding color accuracy, they rate the Phase One back slightly higher than the Nikon, so you have no quarrel with them there, either.
Regarding dynamic range, let's see the proof for your assertion. In the real world of photography, I can tell you this. When you're looking at sensors with a 14 stop exposure range (give or take), the real limits on exposure range are the amount of flare in the lens and the amount of light scattering in the camera body. That's what's going to determine, practically, what your exposure range is. I wouldn't even want to guess how a matchup would come out in this case.
But… If you're talking about laboratory tests intended to measure the sensors (not the total system performance range), I can trump your hypothetical with real data. I've been able to test four different sensors that DxO and dpreview have reviewed. In all four cases, my results agree with DxO, both in the tests and the real-world performance. For what it's worth, there is usually little or no difference between the results that DxO and dpreview report, but in the rare cases where there were, DxO matched my results and dpreview didn't.
So, in my TESTED reality, in contrast to your HYPOTHETICAL reality, I have found that DxO inhabits the real world very nicely.
As for your faux bet, I will take you up on it! There are only two conditions.
(1) You have to arrange to get both camera systems to me and arrange the venue for presenting the work. I don't have the connections to do that and without both of those obviously we can't run the competition.
(2) I get to photograph a range of subjects that corresponds to the aggregate number that DxO generates. That includes a substantial number of lowlight photographs, because that's one of the factors that goes into their aggregate result. And I can pretty well assure you that in that quality, medium format backs are not of sterling performance.
If you think the aggregate number that DxO generates is meaningless to you, then don't use it! I don't use it! It doesn't correspond to my mix of photography, any more than it does yours. It's for people who aren't competent to read the individual sensor scores and understand what they mean. It's not for people like you and me who actually know enough to be able to read the more technical results.
Don't castigate them for providing a dumbed-down answer. I can assure you, sadly, that the vast majority of photographers are technically and scientifically illiterate; over the years the articles of mine that have generated the most misunderstanding and misrepresentation by readers are the ones that have included the most complete and careful scientific results and procedures. The majority of people just can't understand that stuff.
It's a damned if you do, damned if you don't situation. The geeks jump all over you if you don't lay out every detail like it was a paper for Nature. The majority of the readers will be totally lost if you do. DxO at least provides a range of level of details; if you choose to drill down to the levels you can get as complete an understanding of what they do as you'd like. They just don't force it down your throat.
pax \ Ctein
[ Please excuse any word-salad. MacSpeech in training! ]
======================================
-- Ctein's Online Gallery http://ctein.com
-- Digital Restorations http://photo-repair.com
======================================
Posted by: ctein | Friday, 30 March 2012 at 04:55 PM
So much pixel peeping or magic dusting of pixels so far from what really matters.
The real world relative difference between DXO scores is so much less than what you do that it matters not so much. You are a much bigger issue in image quality that the difference between DXO scores and the camera you may want or desire.
While it's true great equipment is great equipment and will produce the best images, it's also very much true, great photographers will produce great images in-spite of their equipment.
Go out and shoot, learn to see and do pre/post very well. You will benefit from better images even if your equipment's DXO score is not near the top. Learn to compensate. Ever the best DXO score equipment has it's limits.
Cheers,
Robert
Posted by: robert harshman | Tuesday, 03 April 2012 at 12:47 AM
Canon had a 1-stop high ISO weakness for quite some time.
The 1Ds3 was 1 stop weaker than the D3, and this wasn't remedied until the 5D2 hit the market. On APS-C/FX, the D70 outperformed the 40D and 50D for the same reason.
Afterwards, Nikon had better sensor technology with the D3s, and Canon couldn't catch up until the recent 5D3 and 1DX. The 7D and 1D4 were all inferior to the D3s, despite being released around the same time or later.
On APS-C, the 600D, 60D, and 7D have all been inferior to the D7000.
With regard to this issue, Canon either doesn't seem to take Nikon seriously, and as it was the market leader, or finds that it's lacking in decorum to drive Nikon out of the market, especially since Nikon is a smaller company in market capitalization.
Nikon mainly has the camera business to itself, while Canon has a bunch of other highly profitable businesses, such as printers.
As of right now, Canon is behind Nikon in that its sensor architecture is inferior to Nikon/Sony's at low ISOs, due to extremely high read noises.
The key thing here, though, is that Canon is better than Nikon when it comes to the lens line-up. Nikon has been remarkably unaggressive when it comes to up-to-date VR lenses, and key budget items have not yet been replaced. On the top-end of the Nikon and Canon lens lines, Nikon and Canon are about equivalent, but at the prosumer and consumer ranges, Canon tends to have an advantage.
The IS 55-250mm is better than the Nikon 55-200mm by having an additional 50mm of focal length, while the 28-105mm IS USM beats the socks off Nikon's equivalent offering at 28-120mm.
When Canon loses that advantage, it will need to fight desperately.
Posted by: Inst | Tuesday, 10 April 2012 at 04:31 PM