DxO Will Answer Your Questions

DxO has issued a press release to photographic websites this morning containing two major announcements. First, "as early as next week," it will add measurement results for many new cameras—including some medium-format cameras.

Second, it has instituted a new "Questions & Answers" section at dxomark.com. This will be an ongoing service, but, naturally, some of the most basic and pressing questions have been answered in the first go-round (such as, "How is DxOMark Sensor computed? What is the formula?"). If you've been following the new website and have questions about how the evaluations are arrived at, you can get the answers straight from DxO.

Here are a couple of samples from the first round:

"How can I know if two cameras are noticeably different?"
Consider a digital gain of one f-stop, or similarly underexposing 1 EV at capture and then adding 1 EV at RAW conversion. This operation leads to a loss of 1 in Dynamic Range, 1.5 in Color Sensitivity, and 1 in log2 (Low-Light ISO), corresponding to a loss of 15 in DxOMark Sensor. Therefore, you can consider that a 15-point difference on the DxOMark Sensor scale is roughly equivalent to a difference of 1 f-stop, and that a difference of 5 is equivalent to 1/3 f-stop. DxO Mark Sensor differences below 5 can be considered as not noticeable.

"Should I interpret DxOMark Sensor results depending on the way I use a camera?"
Before evaluating a camera, you should indeed define your own photographic use-cases. The print size that you typically prefer will determine the minimal resolution that you need. Once you know your required resolution, then you can use DxOMark Sensor to look at the best digital camera within the resolution category that you have selected. If your main use case is to view pictures on screen at 1:1 scaling, then you are less concerned with resolution (so long as your screen has fewer pixels than your camera), which means that you can use the "Screen" measurement that dxomark.com provides.

Many more answers are provided at the website. As a gratuitous editorial comment, I'd just like to say that DxO is really to be applauded for providing this very valuable website and service—a perfect complement to sites that test cameras as integrated devices and don't attempt to isolate the quality of the sensor.

___________________

Mike

Featured Comment by Ctein: "I am in geek heaven."

Comments

[Dennis Allshouse]

Good for DxOMark!! Much more satisfying to read some of the explanations. On first read the examples in this article come off a little stilted. But when I checked out the material on the site, I found a lot more information and detail. Clearly the folks at DxO have invested thought and time in responding to the feedback that they've gotten from the photo community.

[Archer Sully]

I found that second answer to be downright silly. IMHO, the way to use DxOMark is to use the comparison function to evaluate the relative strengths of the different products, and ignore the final number entirely. Oh, and remember at some point it is important to consider lenses, flash, etc... Basing the judgement of "best camera" on a single number is .

[dale moreau]

I still am concerned about assigning numbers to subjective views. I will again refer to empirical anecdotes.

I read an interview with one of the Steinway family, Steinway is always a good "art" reference. He said that as the guy in charge of the Artist touring piano inventory in the '40s and '50s. he used to walk with Horowitz to select an instrument for his tour(before he traveled with his personal piano) and he would try a piano and say, "No, no that's a Rubenstein piano" until he found a suitable instrument. The funny part is Rubenstein would do the same and say "No that's a Horowitz piano" until he found one he liked.
So maybe the criteria should be that we choose a sensor by the far more accurate assessment of its artistic results.
That way we could have an Avedon sensor, A Karsh Sensor a Cartier-Bresson sensor an Adams sensor etc. That we we would have only ourselves to blame when the pictures weren't as good as theirs.

dale

[Thomas]

I'm still puzzled as to how the DxOMark Sensor score is calculated from the three metrics. The Q&A says that it is derived from a formula that balances the three metrics "equally". In the page explaining the DxOMark metrics, it is said that the DxOMark Sensor scale "aggregates" the performance values of the three metrics. If so, how is it that the DxOMark Sensor score for the D3 is 80.6 while that of the D700 is lower at 80.5 despite the fact that both cameras' scores for Color Depth and Dynamic Range are identical while the D3 scores lower (2290) for Low-Light ISO compared with the D700 (2303)? Would be grateful if anyone could explain.

[Ctein]

Dear Thomas,

Did you note that the DxOmark site is QUESTION and Answer? Why not ask them????

You're reading this stuff to three significant figures; you're reading it WAY too closely. Meaningless precision.

pax / Ctein

[Dennis Allshouse]

DxOMark does point out that a difference of 5 between two sensor numbers amounts to a 1/3 stop difference. Hence less of a difference indicates an essentially indistinguishable difference. Whew! ;-)

[EE]

ctein wrote:
"You're reading this stuff to three significant figures; you're reading it WAY too closely. Meaningless precision."

Yeah, meaningless precision. But then I wonder why they give their ratings with a sub comma [i.e., decimal point —Ed.] precision.

If a score difference of 5 corresponds to a sensor light gathering difference of about 1/3 stop, then there is no reason to be more precise than this same 5 scores.

Looked at it like this, scores like 80.5 vs. 80.6 are just silly. They indicate, that the sensors were really measured as precise (and accurate), when in reality the measurements problay are more like 80 +/- 5 or something like that.

But to state Thomas' position again:
a) The score is calculated by a formula out of three subscores.
b) In all subscores, a camera is equal or better than another one. By any logic, this camera shouldn't get a lower final score given a) . Anything else is--at least for me--not understandable.

Then, all these subscores are probably not precise enough that sub-comma values are justified. If you calculate (e. g. multiply) three subscores that do not have the precision of three digits (two before and one after the comma) then the final score is even less precise.

So in reality, it would probably be better when DxO gave ratings that are based on a range, e. g.

Camera A has a DxO score of 78 to 83.
Camera B has a DxO score of 77 to 82.

Now it gets really clear, that
a) in reality, they score equal
b) it depends ;-) (e. g. from cam to cam, even cams of the same make)

[Thomas]

EE has stated my position more clearly than I had. I fully agree. To be taken seriously, the DxOMark score should at least be logical. The oddity in the D3 and D700 scores is not an exception. There are others in the database if you start looking more closely.
I've taken up Ctein's suggestion and emailed DxO for an explanation. Let's see if I get a response.