Comparisons Are Odious

I feel vindicated.

Science has proven what I've been saying for twenty years. Boo-YAH.

In a brief, six-paragraph article that I think I shall treasure always, the Research Digest Blog of the British Psychological Society presents a précis of a 2008 experiment by Christopher K. Hsee, Yang Yang, Yangjie Gu, and Jie Chen called "Specification Seeking: How Product Specifications Influence Consumer Preference," from the Journal of Consumer Research.

You can (and should) read the six paragraphs yourself—and perhaps even the full study (it costs $10), but the upshot of the research is that people will choose better specifications even when it contradicts their own experience of quality. Specifications, they say, "exert a powerful, irrational effect on consumers' decision-making, even overriding the influence of a person's direct experience with a product."

Irrational. I love it.

When Hsee et alia asked test subjects to choose a camera based on the image quality of sample pictures, most chose a camera that made more vivid pictures. But when told that the other camera had more resolution (more megapixels), "many more participants chose the camera with higher resolution."

In other words, we let single-index specification numbers distract us from our actual visual tastes.

I knew it.

The final paragraph is worth quoting in full:

The researchers concluded with some advice for consumers: "First, they should seek experience, not just numbers," Hsee's team said. Secondly, the researchers advised people should avoid direct comparisons between products, remembering that your real-life experience of a product will most likely be in isolation rather than side by side with its rivals.

Not to say I told 'em so, but Yr. Nt. So Hmbl. Svt. the ol' empirical photographer has been saying as much fer years....

(Okay, carry on pixel peeping, and ignore my gloat and glee. I think this is maybe just a personal thing, and maybe not as relevant to others as it is to me.)

____________________

Mike  (Thanks to Jay Smith)

Send this post to a friend

Featured Comment by yischon: "Dr. Hsee has the article for download on his University of Chicago web page."

Comments

[Brian]

This is so true about everything that we spend our money on. From our homes to our cars to a bottle of wine. Trying to sort this out for a digital camera has made it a nightmare for me and I am beginning to think that I would be happier with a lower end DSLR and replace it in a few years rather than going too high end now with so many changes going on. I do worry about how long these digital cameras will be usable due to software and storage issues

[Matt Davids]

Must be the only reason the a900 is selling well. :)

[Clayton Lofgren]

I am guilty of paying too much attention to specs, but I am getting better. Before deciding on a camera, I go look at one of my better 8x10s. It once shared space with four other shots on a Mavica floppy.

[Miserere]

Quite right, Mike! Go ahead and celebrate, but know that nobody will take notice of this study or change their decision-making processes. Either people "get it", or they don't; I don't believe in education in this particular scenario.

Personally, I'd like to request membership to your club. What do you call it? "Pictures Not Numbers Club" has a nice ring to it. As proof of my worthiness, I present exhibit A: I've had to send out my K10D to get fixed...a 2nd time; because I don't want to be without a camera another 2-3 weeks I decided to order a backup. I could have bought another K10D or sprung for a K20D given its falling price. Instead I've picked up a K100D Super. Yes, it's a 6MP camera, but it's different to my K10D and will offer me options the K10D doesn't: ISO3200, smaller size and lower "visual impact footprint" being the main ones.

OK, I'll stop blowing my own horn now.

Mike, please e-mail me for my snail-mail address so you can send me my member's card.

[Joe Toole]

I learned a similar lesson years ago when I was a professional musician. I saw many (myself included) fall into traps by going by what we think that the bass, treble, midrange, etc., should be set at on our amplifiers -other than what our ears are actually telling us. I used to call it "playing by eye."
In photography, many of us oftentimes mistakenly assume that having a technologiclly superior machine will constantly produce results better than other cameras. Call it 'photography by hype.'
Sure, technological advancement is a good thing, but the marketing of some of these 'improvements' will have you believe that you can't possibly live without them, when in reality, you're just splitting hairs.

[Tim]

Seems reasonable to me. I was very happy with my 10MP D200, allegedly 12MP G9, Hassie and 5x4" combo last year. This year I gather there are mystical D300 and D700 and D3 things out there.

But I still get nicer tones and pixel-sharpness out of the D200 in one shot than I do from the G9, so I for one will not be reducing image-quality to just a number.

[erlik]

No, not relevant only to you. :-)

The citius-altius-fortius [swifter, higher, stronger —Ed.] of camera marketing has become largely pointless.

[Neutralday]

The conclusion for camera manufacturers will undoubtedly lead to a 21 megapixel Canon G11.

[Mark Roberts]

I've seen the same phenomenon for years. When I worked in a high end audio shop (Linn, Conrad-Johnson, etc.) years ago we had a customer shopping for speakers. We happened to have a pair of the competitor's speakers he was considering, so he could compare both sets of speakers with identical source components. The speakers we had were clearly better and he said so (there was really no comparison, in fact). But he later bought the competing model anyway. Why? In the notoriously specification-contemptuous world of high end audio, the specs weren't a factor. But the competitor gave a better "technical" explanation of *why* their speakers were better (linear, phase-compensated oxygen-free standing wave nodes or some such), whereas we just relied on listening tests and the fact that our product was superior when judging by listening.

This happened enough times (and doubtless many more times that we were aware of because most customers wouldn't admit it) that it became obvious that is wasn't an isolated phenomenon: People make irrational buying decisions and then justify them by rationalizing technical details.

I have entirely given up on on-line review sources like DP Review and similar sites. The information I get from T.O.P. and Luminous Landscape is infinitely more relevant and useful. Unfortunately, neither includes numbers and graphs with which I can impress my co-workers with the superiority of my purchases. I suppose I'll just have to live with that.

[Kainnon]

um... Yeah. It's not particularly irrational, actually.

I don't see how being "swayed by the numbers" is irrational. If anything, numbers are arguably more often justified in may areas of life as a rational basis of decision! ie: Can't get a loan, cuz of your credit score! Wear a jacket, it's X degrees outside!

Especially numbers with units are particularly "rational" in our modern "scientific/digital-age". Megapixel "units" that have been imbued with such importance and significance by advertisers for ages (in the digital world). If anything, subjective evaluations are irrational, and temperamental. Measurements are "rational". And more often, we're under the implicit persuasion that they've been made by "experts" and the experts have more knowledge about such things than we do. Thus relying on specs, is by extension a reliance on experts (or advertisers in disguise).

While buyers "[...] should base their purchase on their direct experience rather than specifications" in the cases where differences in specifications can not be perceived, I venture that the ppl making Bose sound system will be coming out with a camera, soon enough.

Irrationality is nothing new, in the world of psychology. Heck, everything we in psychology do in psychology is show how ppl are irrational. The only ppl who listen to us are advertisers, unfortunately. Oh, and the occasionally court/judge/lawyer. For some reason, they're under the belief that for the most part, ppl are rational.

BTW, that journal isn't "published" per say. It's only online, not actually "on paper" which is a hit on it's credibility. Furthermore, the study was on a population of Chinese students. Not to rag on the Chinese, but they're a more "collectivist" culture, which may impact on their willingness to take a stand on subjective issues and appraisals. In my opinion, this study on specification seeking is just an extension of social conformity studies. Have a look up on Solomon Asch, and his studies on social conformity.

Truth is subjective, it turns out. We're just too used to it being tied down with a shared reality.

[CP]

Kamusta Mike, Osashiburi desu (not that I have met you actually). But long time no post in this case.

My grandma always say - experience is always the best teacher. When it comes to photography and cameras, not all of us has the resources and time to have that. That is why we always check "what Mike J. has to say" first and other likes. Unfortunately, some "other likes" dwell completely with what the article actually pointed out and we always look for the "highly recommended" seal of approval. Now that the A900 was "featured" here, I think I want one even if I still have a good splash proof K10D that works perfectly well for a geologist's work.

[David]

Buying shiny new gear is a lot of fun.

[ben ng]

The Magnum blog http://blog.magnumphotos.com/2008/11/wear_good_shoes_advice_to_young_photographers.html
on advice to photographers has a number of great photographers giving their two cents' worth of advice, with not a single mention on what equipment to buy...I personally like being reminded "to wear good shoes."

[Robert Roaldi]

Kainnon,

Just a quick about online publishing. I'd say the credibility is determined more by the peer review requirements and the quality of submission, not necessarily whether the journal is actually printed on paper. It's a growing trend in scientific publishing to go "online" only and never to print on paper at all. I work for a scientific publisher and the costs of printing, mailing, etc. are going steadily up, while subscriptions are going down. The libraries of the world can't afford to house more paper nor the upkeep that it requires. That industry is changing rapidly, the rules are changing every day.

[Ken Tanaka]

Hardly surprising but nonetheless fascinating results.

[john robison]

Exactly why I buy used cameras. As I have mentioned before, my first DSLR purchase 4 months ago was the now cheap Oly E-300. With the prices still falling I think the next body will be the positively ancient E-1. That said, the micro 4:3 announcement does have me intrigued. Haven't been able to check out a Panny G-1 yet to find out if their EVF contraption really works. If it does, this may drag me to my first new camera purchase in 35 years.

[David Dyer-Bennet]

I wish the sides could be less divided. Pixel-peeping is *good* -- when employed as a way to understand imaging. It's bad when it's used as an *alternative* to understanding imaging.

Read Ctein's articles here; he's very solidly grounded on what makes good images (paper prints, even), but he's committed to understanding how different hardware produces different images.

[Yuri Wolf]

Mike,

I have heard somewhere that the photographer has much more impact on the resulting image quality than the camera. If this conjecture is true, wouldn't it make shopping by camera specs much more rational than by picture samples?

[bobdales]

Mike,
Perhaps you can title equipment ruminations "Seeking Experience".
bd

[Eolake Stobblehouse]

Congratulations.
It's amazing how powerful vindication feels, isn't it. I've been saying for years that a real estate crash was coming. I said it for example to my sisters when both of them independently tried to convince me to invest in real estate a couple of years back.

So now that it is here, I'm astounded that it makes me feel *good* to be proven right, even in the face of the misery the crash is creating.

(Although not as much as it would have created had the house prices continued to rise 15% per year. It *had* to burst. And they are still overvalued, it will fall more.)

[Eolake Stobblehouse]

Damn, I'm just realizing that this phenomenon surely also works in my business. I sell girly-pictures (the "joyful nudes" banner on this page), and my site has five updates per week for members. I think this is about as much as anybody can really keep track of. But some of my competitors has three updates *daily*! and they are much richer than I am. The spec is higher. (Also one of their updates can be a gigabyte, whereas one of mine is maybe 20MB.)

[Scott W]

I'm downgrading from a 5D to 6 MP DSLRs and just avoiding low light, or accepting blur... Well, um, maybe I'll keep the 5D and just buy a lot of old DSLRs as "backups".

I like buying old, well-handled equipment. What can I say?

[John Roberts]

Reminds me of the article in Time magazine several years back announcing the groundbreaking discovery that men and women really are different. Again, we have research to prove the obvious.

To me, it's readily apparent that there are basically two groups standing under the umbrella of photography (especially online): those whose obsession is photography, and those whose obsession is photography equipment. At first there appears to be a lot of crossover between the two groups, but if you read the posts, and view the work, it rather quickly becomes apparent onto which side one falls. Those obsessed with equipment will always be chasing the numbers, and if that's what thrills their soul, so what?

[Luke]

"First, they should seek experience, not just numbers."

Correct, and pixel-peeping IS EXPERIENCE.

Did you not choose which film to use by LOOKING at the results? Or was any old film OK?

I want to know what EXPERIENCE to expect before I shell out $4000 for a new sensor.
Does it focus accurately? Is the meter predictable? Is it responsive or sluggish? Are those pixels blotchy or clean? A good review tells me about the EXPERIENCE.

I can't afford NOT to pixel-peep.

Please don't confuse pixel-peeping results with reading specs and conjecturing endlessly in the forums.

Thank you.

[jose g]

I always knew you were right. If only because I was in full agreement.
But it is not too bad applied to cameras. Compare: Consider cars... How many people buy what they really need? How many buy enormous and expensive gas guzzler to conmute a few miles every day? When you are forced to drive at 55 mph, who needs a much faster car, except profesional racers?
Etc.

[Gary D. Thompson]

Buying a product based on it's technical capabilities vs a sample output is not irrational as long as there is no know data conflict or flaw.

One can certainly "over buy" this way, but you do have "the meat" of the product to use at your and it's fullest advantage going forward.

GT

[Karsten]

"but the upshot of the research is that people will choose better specifications even when it contradicts their own experience of quality"

I must say that I'd be more impressed by the result if the "own experience" weren't limited to being shown a couple of pictures that looked more vivid. It would appear that the test persons did not have that much experience in digital photography because to prefer one camera over another based on "vividness" of the pictures does not strike me as the most rational of positions. Moreover, does the study show that people are swayed by numbers or by vividness? What I mean is that "sway" depends on your base case. Take the choice based on resolution as the base case, and people appear to be "swayed" by vividness. What is more rational?

Karsten

[john robison]

Re; John Roberts observation of the "...basically two groups.." Come on John, don't forget the 'forgotten camera nuts' like me. Of course, after some consideration I suppose I fall into the second group, but 30 years out of date.

[Randolph]

This is one of the big differences between a pro and an amatuer; pros know what the numbers mean, and how much weight to assign them. One's intuitive response to a technology, especially one one is exposed to for a short time, is not always reliable--some things look good for only a little while and the numbers can sometimes show you that. In terms of cameras, I think we are looking at the wrong numbers. What we want is a number something like noise/pixel/second x resolution, and that would give meaningful, though not complete, information for comparison. I suspect that camera designers actually know quite a bit about this, and what they know is not mostly published. :-(

[David Bostedo]

I agree with Karsten on the viewers limited "own experience". In this case, perhaps the viewer of the pictures knew they were inexperienced, and didn't trust their own judgement on the matter. Maybe they were afraid that they didn't really know what they were looking at, and that the specs given might lead to better pictures once they learned. And from the "more vivid" comment, it sounds like maybe they were correct.

And from what I recall, the reverse of this tends to happen with televisions (or used to, before digital internet specifications). People used to tend to buy the one that looked "more vivid" in the store, hence the reason the stores tend to crank up the brightness and saturation on their TVs. I wonder if that's changing now that you could have a bit of a "spec" war in the TV world too?

[Kainnon]

Robert Roaldi,

I point out the issue of it as "online" and not really "published," was because as an unofficial publication, my scholar searches didn't actually find it the first few times. I needed to actually go to the Chicago Journals site, and pull it from there. Though, now I see that it was available from his own

Thus, I question weither how "review-able" and accessable it is, by other researchers, when looking up similar articles through the (paid/subscribed) world of scholar work.

[Robert Roaldi]

Kainnon,

It's a frustrating thing sorting out the good from the bad on the web. Over time, I hope, things will become more clear. Informal sites may always be a gray area, though, with "self-published" respectable publications mixed in with the cranks and no obvious way for a newcomer to tell them apart. Without a brand-name logo in the corner, it is difficult to trust what you read.

In the days of paper subscriptions, the cost of producing the output was a severe impediment to cranks and we weren't bothered by them much. Now they can just click on a virtual button, almost no cost. The web was supposed to democratize knowledge, but free does not always mean credible.

[Nathan Smith]

Yuri, perfectly said.