DxO Adds Medium-Format Digital Cameras to dxomark

DxO Labs announces today the addition of raw-based image quality data and DxOMark Sensor rankings on its popular www.dxomark.com website for an initial set of high-end professional medium-format cameras: the Hasselblad H3DII-39, the Leaf Aptus 75s, the Mamiya ZD, and the Phase One P45+.

Photo enthusiasts, professional photographers, and the media now have a valuable tool with which to compare the RAW sensor performance of these medium-format cameras with high-end, high-resolution DSLRs. How is it possible to make valid comparisons between cameras with widely disparate sensor sizes? It's possible by reviewing the cameras' rankings for each of the three separate metrics that make up the DxOMark Sensor scale (Color Depth, Dynamic Range, and Low-Light ISO).

For example, if medium-format cameras do not receive top marks on the overall DxOMark Sensor scale because of their inherent Low-Light ISO limitations, DxO Labs has found these models' Color Depth and Dynamic Range performance to be very striking when compared to high-end DSLRs. Also, despite the clear challenge from DSLRs across all DxOMark sensor metrics, medium-format cameras still lead the way for large-print photography because of their very high resolution performance capability.

(From the press release.)

A sample comparison:

This sample would indicate that there is virtually no difference between the sensor quality of these two cameras, apart from low-light capability (which is not a limitation for the Hasselblad's target market, since studio photographers use artificial lighting. Might be important for you, though). Of course the big difference is image size (39 MP vs. 12 MP). Interesting, as I would have thought that the medium-format camera would have had better dynamic range than it does. I thought that was the tradeoff with high-ISO capability. Guess not....

Mike

Send this post to a friend

ADDENDUM: Read Michael Reichmann's thoughts on these new numbers. Michael's actually used a lot of medium-format digital cameras, and, like him, I tend to trust eyes over measurements.

Comments

[jay moynihan]

"DxO Labs has found these models' Color Depth and Dynamic Range performance to be very striking when compared to high-end DSLRs."

Hmmm. After doing a, albeit quick comparison, (phaseone/FF-dslr/D200) I would suggest "very striking" to be hyperbole. Perhaps "measurably better"? :)

The press release also mentions resolution. I would assume that would be the "striking" advantage, and DXO does not, I think, measure that.

[Nick]

Mike, just in case you missed it; Luminous Landscape posted their take on the DxO tests of Medium Format backs earlier today. Michael is not impressed.

[Martin Doonan]

I suppose this comparison comes from the fact that this is a per pixel comparison, rather than a per square inch of print comparison. Therein lies the limitation in comparing different sensor sizes (would also go for "full-frame" 35mm versus APS).

[Jean-Pierre Grüter]

I work with the leaf aptus 75 digital back and a nikon d3. I prefer the color of the nikon and the dynamic range is higher (ISO 200, Raw-files). As Mike said the only advantage of the present digital backs is resolution. The Raw-files from the d3x are also very good, so I would like to have a d3x senor in the MF-size (45 x 56 mm). That would give a 70 MP digital back with the best color rendition and dynamic range at the moment and very good high ISO performance as well. Nikon as a big company is able to investigate a lot in the control of the chip production and the data processing. And I wouldn't be astonished if nikon is producing a MF Sensor in the next few years.

[Pascal Scheffers]

When you use the compare function on the DxO website, the SNR, Dynamic Range, Tonal Range and Color Sensitivity tabs all provide a 'print' performance measurement. This seems to normalize their measurements and appears to include the sensor resolution into the cameras performance.

When you do that for the D700 and the Hasselblad, the graphs actually show that the Hasselblad performs better at low ISO than the D700. It is half a bit here, a bit there but if you look at how other cameras compare these things seem to add up to significant differences in image quality.

Comparing cameras without taking input and output resolution into account is flawed, I don't understand why graphs on the DxOmark website default to it. Their overview results take them into account.

[Rob]

Ever since I first looked at the DxOmark site I thought it was really silly that it did/could not include resolution in any meaningful way.

I assume that it would be necessary to convert the raw file to come up with an appropriate printed image that would be able to show the resolution differences between various cameras. It would probably be necessary to print to various sizes in order to see how resolution differences kicked in.

Lets not pretend that a 12mp D700 will have equal resolution to a Phase One 60MP back, or even a 24mp Sony D900, at print sizes over 19x13. And surely such differences will be very important in some buying decisions. A 1.4x, or more, increase in resolution is far more important than almost immeasurably minor differences in colour depth, tonality, dynamic range etc.

[erlik]

As Mr. Reichmann noted, there's something iffy with these numbers. Particularly the part when they measure with noise reduction already on in Nikon and no noise reduction in medium format.

Besides, what kind of ISO sensitivity test is the one that shows Hasselblad has just the sensitivity of 50 and PhaseOne of 50 and 100? Are they lying so much? How can people work with no increase in sensitivity?

But this will "show" that 35mm digitals are better than medium format. Oh goodie.

[Ken Tanaka]

Throughout my business career I got many good chuckles watching people celebrate hard figures congruent with their opinions but deny and discount them when their beliefs and observations were threatened by the same figures.

Michael Reichmann gave me just such an ol' time chuckle this morning.

To a limited degree I agree with him that DxO's ratings are limited in scope and do not necessarily accurately portray the total performance of cameras (or, now, digital backs).

I do, however, believe the DxO figures are more reputable than Reichmann's current dismissal merits. The DxO methodology is consistent in measuring the three qualities in its sphere. Contrary to his criticism, that methodology is designed to be independent of sensor resolution and sensor format; it measures what it sees. As the DxO tests are the only consistent and independent test suites for camera sensors they are worth at least noting.

I have not used all of the cameras or backs that Reichmann has used; few photographers have. But I can say that DxO's relative figures for the top-end Canon's, the Leica M8, the Canon G10, and now the Mamiya ZD are very consistent with what I observe. I generally discount the ISO performance, since the top Canons' performance in this metric is so far out of the league of the others. But DxO's color depth and dynamic range figures have been quite reasonable, again when used as relative measurements among cameras.

For Michael Reichmann to claim that the DxO figures place medium format digital backs "under the gun - wrongly" is a little overly defensive (and the source of my aforementioned chuckle). Look, let's be fair and honest just for a moment. The über-expensive mega-resolution medium format digital backs are principally aimed at one relatively small niche market: advertising photography, particularly the rental and leasing businesses that serve this market. They are largely products of craftsmanship engineered to produce excellent results but under far more limited conditions than what a normal dslr experiences.

In contrast, cameras from Canon and Nikon are products of many orders of magnitude more research and engineering investment. Both of these companies have far more experience with digital imaging products and sell more units in a month than all medium-format digital backs that are sold in a year. Why wouldn't we expect them to test so well relative to medium-format digital backs after so many years of experience and development?

No Bentley owner wants an auto magazine to claim that a Lexus outperforms it in most practical categories. But it's probably true.

[David Dyer-Bennet]

If carefully run visual trials (multiple subjects, double blind, controlled lighting, and I'm not sure what all else) give different results than "measurements", then you're measuring the wrong thing (or measuring it incorrectly), certainly, if you're evaluating imaging equipment for artistic / graphic use.

But "golden ears" is still a term of scorn and derision for me, and Absolute Sound I have always understood to be a kook publication; Michael Reichmann's references to that stuff over on LL drastically reduce my ability to take what he's saying about camera evaluation seriously. (The incredible difficulty of getting anybody claiming golden ears into a controlled testing situation was the clear-cut evidence they were kooks.)

I've heard the claims about contact prints vs. enlargements, or different appearances to medium format vs. 35mm, and except for blatant issues like grain size, I've never been able to verify any of them. But I haven't spent a lot of time around large-format photographers. Also -- just when I was starting to think maybe I *did* see such differences, I got some 8x10 snapshot prints that Ctein made from my negatives, which were startlingly better than most 8x10s of anything I've seen in my life; so there's another big variable not to forget, the quality of the printer, by which I mean the person!

[hugh crawford]

In film terms it's like asking which is better, 35mm pan-x or 8x10 tri-x

If you read the DxO website as the equivalent of testing film emulsions, it's useful.
It is less useful for evaluating total systems, and DxO doesn't seem to take into account how artifacts scale from one format to another with anti aliasing filters etc.

If I am mostly making photographs of dark gray featureless walls and step tablets, the DxO website is all I need. If I am going to put a lens on a camera and photograph something more complex , I would need to think about it a lot more.

As someone who has had more than one instance of a really ugly depiction of a dark grey featureless wall ruin an otherwise ok photo , I think the DxO website is pretty useful, but the DxOMark taken alone is sort of useless.

[Mike Johnston]

"Lets not pretend that a 12mp D700 will have equal resolution to a Phase One 60MP back, or even a 24mp Sony D900, at print sizes over 19x13. And surely such differences will be very important in some buying decisions."

But surely you can figure that part out for yourself?

Mike J.

[Dennis Allshouse]

I can't help myself: let's recall that Mr. Reichmann recently hosted a comparison of the Canon G10 and some MF digital back and found that, what, half the time people could not distinguish between the two!! Beware the GoldenEye, the claws that grab,the teeth that bite.

It seems to me if you print large then the bigger chip will matter more than if you print moderate. And I will stipulate that well practiced eye can see technical flaws/quality flaws. On the other hand at certainly at web sizes you can't see a difference and if you think you can you are bullshitting yourself.

This goes back to the sharpener/leveler discussion, doesn't it?

[Marlon]

There is a bit of absurdity to these tests don't you think. The fact that resolution isn't a factor it clearly gives an advantage to lower MP camera's. Also, Nikon, cooks there RAWS right on the chip to achieve better noise compression. In these tests they get rewarded for this. All the MF's and the A900 seem to get penalized for not doing this, which is another flaw in the test.

[Marlon]

At Ken:
MR is correct with his assertion, the medium format camera's with there ungodly resolution and lack of AA filter completely smoke anything that Canon or Nikon is offering. The level of detail produced from these "uncooked" sensors is so visually noticeable that any person or photographer would notice it immediately side by side. This along with the specially designed better dissolving lenses of the MF camera's is what sets them way apart from 35mm DSLR's.

Honestly, rather than you chuckling you should really go out and take a look at print made from the top 3 backs tested and then compare it to the best of what is offered at 35mm. Instead of calling the backs impractical, why not interview a professional fashion or high end landscape photographer why they choose to use these tools over the more "practical" 35mm systems. You might be surprised by what you hear.

[robert e]

Shorter Michael Reichmann: Size matters (As in physical area, not number of pixels.). And I think he's right.

It's true that one big difference between the two sensors above is "resolution" (number of pixels), but an even bigger (excuse the pun) and, I suspect, a more significant difference in terms of image rendering is the size of the projected image being captured. No, I can't prove it on paper, but I am convinced that 20 megapixels spread over 2000 sq. mm is going to give me a different result than 20 MP squeezed into 350 sq mm, even if the photosites themselves are identical and angles of view are the same. Needless to say, the behavior of light and optics is a constant that doesn't scale across sensor formats. In other words: even with digital sensors, it could be that magnification matters

This may or may not be accounted for in the the measurements supporting DxOMarks, but it doesn't seem to be addressed in the literature or discussions about it, either.

Reichmann also makes a reasonable point about how some RAW files are more raw than others.

That said, the angst over DxOMark not being able to tell us everything we want to know about a camera seems overblown.

[fjf]

Isn't Michael Reichmann the one who found prints from MF and Canon G10 sensors to be indistinguishable?

[John Camp]

"I can't help myself: let's recall that Mr. Reichmann recently hosted a comparison of the Canon G10 and some MF digital back and found that, what, half the time people could not distinguish between the two!!"

That's not exactly what he said - essentially he said that under certain conditions, in smaller prints, people couldn't reliably pick the digital back. In other words, if you rig the test in favor of the smaller camera, you have a hard time telling the difference. The same would be true with a MF film back and a Penn F, if you printed small enough. I think this may have been one of Michael's bigger missteps, because he's being mis-cited all over the photography blogs. He was not saying that the G10 was generally a match for a P-45; he was saying that under an *extremely* limited set of circumstances, the prints would be hard to tell apart.

I also think he's correct about the DxO tests. DxO tests a limited set of characteristics and then - whatever small-print reservations they may hint at -- they trumpet them around the net as "comparison tests." No they're not. They're *extremely limited* technical tests, that tell you almost nothing useful that you need to know about a camera that you might want to buy.

JC

[Eolake Stobblehouse]

People may not notice that DxO marks don't include resolution, and may just see the numbers and conclude that MF is a rip-off.

Reichman also claims that lack of noise reduction in raw files in MF is a big issue here:
"In such a comparison Nikon files will show very little noise, whereas the MFB files will appear extremely noisy, and If you just look at the uncalibrated and unprocessed data this converts as well into lower dynamic range numbers – something which is quite obviously not the case in the real world."
If that's so, then that's clearly an important factor.

... Of course then there was that G10/Hasselblad article... Reichman is scitzo, like many of us. :-)

[David Bostedo]

"...and, like him, I tend to trust eyes over measurements."

Mike - If generated correctly, I would take numbers over your and Mr. Reichmanns experienced eyes any day. I would only take the numbers to tell me what they can, and only if interpreted correctly. (I personally hate the fact the DxOMark generates a single number score, since it's sure to be taken as some sort of perfect rating for a sensor.)

So there are definitely some caveats in there, which Mr. Reichmann does a good job pointing out in his article (anti-aliasing, RAW conversion, etc.). I would agree to trust your eyes if there was a test setup like the one Mr. Reichmann created with the G10 vs. Medium format example. However, any knowledge of something beforehand (like what camera was used) would make you biased in this case. I think it's possible that Mr. Reichmann got a little defensive initially because of this.

What I found really interesting, looking at DxOMark, is that the ISO's on the P45 are all generated - done in the RAW conversion. The amplification on the sensor isn't done at all. I also find it interesting that the test results imply that, except for resolution, data coming off the sensor of the Canon/Nikon/Sony set is as good or better than that coming off the medium format backs. The pixels they use are no better than a D3X; The dynamic range means that their 16-bit images may as well be 14-bit.; And the color depth is about on par with standard DSLRs.

That means, to me, one of two things :

1) Medium format really only has a benefit in resolution.

OR

2) There is another advantage in that there's something in the system (no AA filter) and file processing that is beyond what Canon/Nikon/Sony/et al have come up with, and it's allowing medium format to maintain better quality files that lead to better quality prints. (Mr. Reichmann seems to fall into this camp a bit.)

I'm leaning toward #1...perhaps Nikon or Canon should make a 45 MP medium format sensor and we can know for sure?

[Mike Johnston]

David Bostedo,

http://www.nytimes.com/2009/02/03/books/03gewen.html

Mike J.

[Ken Tanaka]

Marlon:
"Honestly, rather than you chuckling you should really go out and take a look at print made from the top 3 backs tested and then compare it to the best of what is offered at 35mm. Instead of calling the backs impractical, why not interview a professional fashion or high end landscape photographer why they choose to use these tools over the more "practical" 35mm systems. You might be surprised by what you hear".

I am a mfdb owner myself and do have some personal and professional insight into the subject. I don't have to "interview" anyone. But you've missed the point of my remarks. Not important.

[David Stock]

Am I misunderstanding something? I thought that the dxo database DOES take into account and compensate for resolution, at least at a "normalized" print size of approximately 8X12 inches. (See http://www.dxomark.com/index.php/eng/DxOMark-Sensor/Data-normalization). If true, this would seem to be pretty useful.

If a camera produces lower quality prints at 8X12 inches, it might still theoretically produce higher quality prints at large sizes. (Due to interpolation issues as well as other things.) But that is a narrower discussion, I think.

[David Bostedo]

Mike - Just to clarify, I totally agree with that article. But that's more about overstating the numbers, or trying to use them in ways they weren't intended, or getting cause and effect wrong.

Those are all things I'm afraid people do or will do with the DxOMark numbers. But, taken for what they are (and assuming the test is set up correctly - which isn't always a given), I'd trust them over someone's eyes.

Maybe we're both saying the same thing, but coming at it from different sides?

[Adam Maas]

DXO proved their tests meaningless when they rate cameras with identical sensors and processing differently for IQ. The tests are meaningless.

See the ratings for the D700 and D3, the Oly E-420 and E-520 (as well as the E-400 and E-500 and E-410 and E-510, all of which share processing and sensors between the comparable 4x0 and 5x0 models) and the Canon 20D and 30D (Same sensor and backend processing, only changes were to the JPEG engine).

[Rob]

""Lets not pretend that a 12mp D700 will have equal resolution to a Phase One 60MP back, or even a 24mp Sony D900, at print sizes over 19x13. And surely such differences will be very important in some buying decisions."

But surely you can figure that part out for yourself?"

Figure out in principle, yes, as stated, but visualise in a comparative way, no Mike. There's the rub . . . how much better (in resolution, tonality etc) is a 20X36 or (19x13) print likely to be from the 24MP A900 or the 12MP D700? This is the issue that will concern many, rather than worrying about the aforesaid invisible differences in Dynamic range, colour depth etc.

Is the 1.4x increased resolution of a A900 going to make an important enough difference over the the D700 at say 24x16 to make it a deal breaker? Who actually really cares about a .01% difference in dynamic range or colour depth?

[Ctein]

Dear Adam,

Identical sensors and CPUs do not guarantee identical results. Noise, for example, is also controlled by heat sinking and dissipation, power supply conditioning and the grade of amplifier used. Along many other things. Unless you know that the entire mechanical and electronic construction for the sensor assembly and associated electronics is identical in two cameras, you don't know that they will perform the same.

Furthermore, the Nikon results are identical to within 1/50th of a stop. The Oly to a 20th. How much precision do you expect? Or need?!

If you think that degree of uncertainty in measurement makes results meaningless, then you don't understand what you're reading.

pax / Ctein

[Ctein]

Dear Folks,

I fear a number of the posters here have ignored these key paragraphs of Michael's:

"Are DxOMark and possibly other numeric tests invalidated? Is subjective evaluation the only appropriate way to judge equipment?

I don't think either is true. There's room for both, but both require that the consumer of the information apply judgment and intellect to the evidence presented. When looking at numeric analysis, such as DxOMark, one has to inquire as to the nature of the things being measured, and appreciate both what has been measured and what hasn't. And when it comes to considering a subjective evaluation one either needs to feel confident in ones [sic] own experience and ability or trust that of the person being listened to.

My fear though is that the web forum fanboys are going to jump all over these DxO results and make inappropriate conclusions."

I trust this need not be belabored?

pax / Ctein

[Ctein]

Dear Marlon,

DxOMark is not a contest. It's not supposed to be fair, it can't give 'advantage' or 'penalize' because that's not what it's for. If you think it is, you're misusing the data.

It's providing comparison data for one performance metric-- RAW image quality, out of camera. How that quality is achieved (large sensor, incredible internal signal processing, Maxwell demons, Pratchett imps....) is of little import to most photographers.

It is no more nor less than that and this is useful to many photographers. If it is not useful to you, don't read the site.

pax / Ctein

[David Dyer-Bennet]

Marlon: did you just say the medium format lenses had higher resolution than the smaller-format lenses? Because this is totally contrary to how things were for the previous 30 years, at least. Can you point me to some evidence for this remarkable turnaround?

[David Dyer-Bennet]

David Bostedo: Sure, people will misunderstand / misuse the DXO numbers. But think of it this way; many of those people were previously using the resolution as their sole performance metric.

[David Bostedo]

Adam - A few things about the numbers :

-- No two cameras are exactly alike. DxO could test several D3's and get numbers within a certain range. So I'd never expect two separate models to come out exactly the same.
-- Differences in the numbers are not necessarily significant. If one sensor scores a 75.2 and another scores 75.6, they should be assumed to be pretty much identical. That small a difference can't be seen and isn't significant. (I can't find a good link to say what IS significant, but it's out there somewhere.)
-- Just because two cameras share the same components doesn't mean you'll get the same exact results. The arrangment of the electronics on the boards, and the boards themselves, as well as differences in manufacturing and packaging, could definitely make a difference. Especially with regard to noise, some of which is certainly due to component proximity, packaging, and heating. These could all be altered from one model to the next.

In short, the differences on the DxOMark site between the D700 and D3 in no way invalidate their numbers or testing. But your statement is what I was getting at (and I think Ctein, on this site before me) in my earlier post. The numbers can be mis-read, mis-understood, or mis-used; But that doesn't make them invalid. It just means maybe they aren't for everyone.

[Dennis Allshouse]

Well, we've spilled a lot "ink" over what is essentially a moot point. People who can afford digital MF, I suspect, don't give a damn about DxOMark and what it means. And for those of us who can't afford the big toys, who cares. We have other fish to fry.

I've never seen a print from DMF so I have no idea what to expect.

Marlon, I hope you have. And BTW all 35's post process in the camera.

So, let's get out there a take some pictures that we do care about. And I for one refuse to be intimidated because I'm not sporting the big package!

[Ben Mathis]

I like the DXOmark site *because* it forgoes megapixels. I assume that people pick their MP number based on their intended print size. Once you know that, you want to compare similar cameras to see how they stack up. I find the sites that include MP ruin it for me, because I care about the image quality at web size, and 13x19" prints (as my maximum size).

[Ken N]

When one of these "Golden Eyes" can identify the camera used to make a print, I'll be convinced.

I challenged a "Golden Ear" one time to identify the type of microphones and mike positions used in a recording of a piano. This individual, who is quite famous and published, swore up and down that it was a pair of AKG 414s at one meter out from the lid.

In reality, it was a pair of Radio Shack PZM microphones taped to the underside of the lid.

When A/B comparing anything, you can see differences and make judgements about those differences. However, when presented with a single print (or audio sample) there is no way for a person to correctly identify the source unless there are specific aberrations one has learned to identify.

[Ctein]

Dear Dennis,

A 5-unit difference is supposed to equal approximately one stop in performance. I can safely assert that a one unit difference is essentially indistinguishable. Realistically, I think a 1.5 unit difference would be of no import.

I am very much with Michael on the problem of spurious precision (the equivalent of "empty magnification" in telescopy and microscopy). I really wish DxO would not give figures of merit as more than integer values and would roundoff ISO equivalencies to the nearest 10% (which would still be an insanely precise 1/7th of a stop).

It's extremely difficult for even trained and knowledgeable people not to unintentionally read spurious precision as real data (check out the history of the human body temperature being supposed to be 98.6°). I don't hold it against anybody who gets sucked in by the meaningless digits.

~ pax \ Ctein
[ Please excuse any word-salad. MacSpeech in training! ]
======================================
-- Ctein's Online Gallery http://ctein.com 
-- Digital Restorations http://photo-repair.com 
======================================

[Ctein]

Dear Ken,

Actually, I can do that, for the very modest number of digital cameras I'm familiar with. In most cases I can even do it without an A/B comparison. They all have unique subtleties to their images.

But this doesn't prove anything, one way or the other. To make those distinctions, I am doing the equivalent of pixel-peeping. If the two cameras are of overall comparable image quality, what I'm talking about are kind of "flavors" which may lead one to prefer one camera over the other, but it truly is a matter of taste and it is in no way quantifiable. Usually it's not even a matter of taste; it's just irrelevant.

~ pax \ Ctein
[ Please excuse any word-salad. MacSpeech in training! ]
======================================
-- Ctein's Online Gallery http://ctein.com 
-- Digital Restorations http://photo-repair.com 
======================================

[Mike Johnston]

"I am very much with Michael on the problem of spurious precision (the equivalent of "empty magnification" in telescopy and microscopy) [...]
It's extremely difficult for even trained and knowledgeable people not to unintentionally read spurious precision as real data (check out the history of the human body temperature being supposed to be 98.6°)."

Ctein,
This reminds me that it can go the other way too...I remember a reviewer who had a five-star rating system, but who routinely gave ratings down to the *quarter* star. I asked him why he didn't just make it a 20-point rating system to begin with, since that what it was, and he said, "too finicky." [g]

Mike J.

[Rob]

Another thing I find very frustrating with the dxomark site are what seem to me to be straightforward anomalies or inconsistencies.

A glaring one, when comparing the Canon 5DII, A900, Nikon D700 concerns dynamic range. On the final tally they are given as 11.9, 12.3, 12.2. indicating that the D900 is the best. Many may not look further.

However, and this seems to me to be a very big however, when you click on the dynamic range tab and look at the graphs, all is VERY different. At 200 ISO, the best for all cameras, we have 5DII-11.12, A900-11.18, D700-11.85 - so the Nikon is clearly better. So where did the other figures come from?

It gets better, move on to 800 ISO and the figures are 10.66, 9.44, 11.18 - the Sony is more than a full stop worse than the Canon. In fact the Sony is a stop or more worse than either the Canon or the Nikon at every ISO apart from ISO 200, and even at 200 it is only minimally (.06 of a stop!)better than the Canon.

The Sony, then, has far WORSE dynamic range both on screen shot and on print at all ISOs apart from 200 than the other two cameras (which are similar). So how does DXOmark award the Sony this bogus (as far as I can see) score of 12.3, misleading everyone?

[Ctein]

Dear Rob,

You're reading the wrong graphs. You're looking at the "screen" (pixel-peeping) graphs. Look at the "print" graphs, which normalize to a standard size output-- that's what the aggregate scores are based upon. The discrepancies are now down in the 10ths of stop range-- not of significance and well within experimental variances.

So, not misleading anyone. Complicated and occasionally obtuse, to be sure. It requires a very high level of technical sophistication to be able to make use of all the data on the site.

pax / Ctein

[Rob]

Dear Ctein,

thank you for your response.
But please look again.

I said "The Sony, then, has far WORSE dynamic range both on screen shot and on print at all ISOs apart from 200 than the other two cameras..."

So, "both on screen shot AND on print".

I used the screen shot numbers, but could have just as easily used the print numbers, here they are:

At ISO 200 the optimum for all three cameras: 11.82, 11.99, 12.15, for 5DII, A900, D700 respectively, so still the Nikon is the best, and these numbers still do not correspond to the overall numbers which gave the Sony the best rating, and my point is the same.

At 800 ISO the numbers are: 11.36, 10.25, 11.49. so the A900 is still more than a stop worse than the other two, and remains more than a stop worse at ALL the higher ISOs.

So what, exactly is your point about the print view? Both graphs are remarkably similar, especially in the trend. The absolute numbers for the print view are a little higher, but make no difference to my claim.

The Sony dynamic range, according to both graphs is worse than both the Canon and the Nikon at ALL ISOs, apart from the optimum ISO 200, when it is higher by a tiny margin than the Canon and lower than the Nikon. At ALL other ISOs it is lower, by half a stop at ISO 400, and by more than a stop at all the other ISO's.

Given that the Sony has a score of 12.3 stops of DR on the overview chart, which is nowhere recorded on either graph, and that this puts it above two cameras which have substantially higher dynamic range at ALL ISOs above 200 (indeed the Nikon at ALL ISOs including 200) This IS misleading. Very misleading.

[Ctein]

Dear Rob,

Again, I tell you: don't bring up the screen-level results. They are not germane.

Do you understand that the spread in agregate score corresponds roughly to less than 1/3 stop of performance, according to DxO? You're nitpicking at minutiae.

The dynamic range score is derived from the best dynamic range produced by the camera, not the average nor the trend. The discrepencies are measured in 10ths of stops. You can't really think that matters.

As to where the discrepencies come from... ASSUMING that that DR score given is merely the best data point on the graph (not necessarily a correct assumption-- it may get massaged in some subtle way) experimental variation would be sufficient. Each model of camera is measured at least twice. So there's a minimum total of six experimental runs involved. If each run were accurate to better than 1/10th stop (a truly superb result), you'd get discrepencies of the order seen in the online data, just depending upon which runs are being plotted in which table.

If you take the time to read the myriad technical papers on the DxO site (which is where I gleaned all this information), you'll find nothing misleading. In fact, it's pretty easy to find out that DR is based on print-scale data and represents the best possible result for that camera.

If you don't like the DxOMark, fine-- don't use it. But please stop trying to discredit it based on an inadequate understanding of what's being presented.

pax / Ctein