« Why Is the Canon 6D so Hot? | Main | A Traditional and Uncompromising Technique »

Wednesday, 30 January 2013

Comments

One of my pet peeves is when the word criticism is used to imply a negative judgement in an artistic or literary context. Being critical of someone's behaviour for example generally does mean something negative, but how are we to define the job of a critic, if criticism is only negative?

I sometimes think reviews should only be done by committees, to avoid the problem of unrecognized reviewer bias. That bias can be very subtle. Some people, for example, like "simple" cameras. But what does "simple" mean? To a news photographer doing four assignments a day, it might mean lots of programmable buttons for a variety of different setups, so that he can go from a quick standup portrait to a police-line shot to a basketball game in a school gym, without reprogramming the camera each time. Just change lenses and push the pre-programmed button: simple. For somebody else, it's a 35mm prime and a camera dial set on "A:" simple. With all the different kinds of photographers out there, with all their different requirements, it's hard for any one person to avoid the biases that develop naturally from his/her shooting interests. It's not that they're being dishonest in the reviews, it's just that one particular aspect of camera design seems obviously "correct" to them -- but (with some exceptions) there usually is no such thing.

One good photography forum where you see this all the time is Luminous Landscape, because it is -- duh -- basically a landscape forum. The sine qua non of many landscape photographers is resolution, so they don't want to hear about 12mp news cameras, or even much about handling problems or one-lens experiments. They want to know about "best" lenses, tripods, sensors, etc. There have been a number of somewhat controversial reviews on LL that have hinted that you're wasting your time if you're not out there with a $60,000 medium-format setup. It's not at all that the reviewer is dishonest, it's just that...that's what he does, that's what he's interested in, and that's what he thinks is necessary to do good work. Whatever "good" means.

The problem with critical reviews is that many people are not prepared to accept a negative opinion on something they appreciate. Nowadays it's all too easy to criticise the reviewers because the www allows people to express themselves anonymously, without facing their opponent. This can lead to some very unpleasant debates.
I keep a non-profit, strictly amateur(ish) photography blog. About a year ago I wrote a rather dismissive post on the Pentax K-01. It wasn't a review: I'm not a reviewer and I don't pretend to be one, neither do I have the skills or experience to write reviews. I had anticipated some criticism, but I wasn't prepared for the kind of insults I got from people who seemed genuinely infuriated about my contempt for that camera. It was an entirely personal opinion, based on the camera's specifications and (especially) its looks, but some people seemed to have taken it personally and felt the need to throw everything at me. Someone even implied that I was paid to write favourable posts about some brands and dismissive ones on others! I was shocked. I felt I was refused the right to have an opinion.
Perhaps the lack of respect for other people's opinions is a price to pay for freedom of expression. Or perhaps it's a sign of the cowardice that lies behind gratuitously insulting someone just because you can't see him/her. Or maybe it's a simple matter of miseducation. The jury's still out.

Criticism can be hard to get used to, but some jobs require that you actively seek it out on a regular basis. I work in software, and at my company we require that any changes to code be reviewed by at least one (and preferably more) other engineers with some expertise on the software being modified. Some people have trouble dealing with this, but the better engineers learn to appreciate the feedback and actually dread getting back a "LGTM" (looks good to me) comment on anything large or complex -- it basically tells you nothing. If the reviewer failed to find anything that could use improvement, you question whether they really looked at the code in detail.

Likewise, for something as complex as an enthusiast or pro level digital camera, it's highly unlikely that the design got everything right for any individual user. A review that failed to find a single fault would make me wonder if the reviewer was biased or hadn't actually spent the time to get to know the camera in-depth.

Good reviews and good reviewers are valuable. For single person reviewers, Thom is one of my favorites, and he doesn't hold back on the negatives, even with a camera he really likes. He finally got around to reviewing the OMD-EM5 a couple weeks ago...

http://www.sansmirror.com/cameras/a-note-about-camera-reviews/olympus-camera-reviews/olympus-om-d-e-m5-review.html

Another reason there may be more negatively critical reviews of people's work (as opposed to a product) is that it is often easier to write about what you don't like, than it is to write about what you do like. One can probably list the things you have a negative reaction to a lot quicker and more articulately than elaborate on exactly why something is good from a critical stand point, rather than from just blatant (emotional)admiration.

In my real life job, I do a lot of reviews of employee performance. In my firm, we have about a dozen categories of competencies and behaviors that have to be rated. We use a 360-degree process, where the employee does a self-evaluation and nominates colleagues to contribute to his/her review.

Sometimes I contribute to reviews, but sometimes I am the lead reviewer and have to figure out how to take a half dozen or so colleague reviews, the self-eval, and my own perceptions and distill that down into a single cogent review for the employee.

We try to highlight successes, but also point out areas of weakness and offer suggestions for improvement.

The point of all of this is that I see a range of colleague reviews for the same person -- some are outstandingly positive with the employee needing absolutely no improvement in any area -- some are negative, some are neutral. Interesting how individual perceptions of the employee under review and understanding of the review process vary.

For the last few years, my role has also been to review the reviews at a higher level, basically to make sure that the scores given for the different criteria are consistent with the accompany text. In many cases a high score is accompanied by text that is contradictory. Joe Blow gets the highest scores for work quality (needs no improvement) but the text says Joe Blow never runs a spell check on his reports.

Reviews are not easy. It is not easy to make reviews consistent across multiple reviewers. It is hard to remove the subjective nature from a review. People, cameras, cars, whatever.

I always used to get a laugh out of the Popular Photography reviews where they summarize What's Hot and What's Not. Sometimes the "Nots" are so far out of left field to be ludicrous, generally poking the manufacturer for not including a feature instead of criticizing the performance of a feature that does not work well.

As a coda to this, once my boss gave me a middling review because I did not speak French. I had already worked for him for a couple of years. It was not a condition of employment. Nor had it ever been discussed before. When he delivered the review and dinged me for this, my response was to ask him, "How do you know I don't speak French?" He never asked and had never spoken to me in French before to see if I understood. He then asked me if I did speak French and I refused to answer. I also refused to acknowledge the review unless the ding was removed.

While I realize there is no tit for tat here, I must completely disagree with John Camp's second paragraph wherein he states the Luminous Landscape (if he's referring to Michael Reichman's site) it either biased in its reviews or touts only the most expensive gear. That cannot be further from the truth.

First, this is an excellent column. In today's day and age, reviews on sites like B&H, etc., can be very helpful. I review almost every piece of gear I buy from them. I tell it like it is, from the perspective of an enthusiast, not a pro.

LL discloses every nuance of possible bias, whether the equipment was loaned, purchased, whether he or his reviewers favor a brand, before they review it, and so on. In addition, in the past few years, he's been using mostly compact P&S cameras with great success. He did a comparison a while back of the then top line G10 (12?) against the Leica M8, and the G almost showed it up.

Does he own a 65 or 80 mp back and medium format gear? Of course he does. He's a professional and makes money from picture taking. He demands the best there is. What is wrong with that?

I Am NOT Saying the Fuji X-Pro1 Sucks...

Now, that was some critical review. But it was not written for me because the camera was not on my shortlist beforehand. I was looking for a button-driven non-DSLR with a largish sensor and low-light capability, as my first serious camera.

The OMD was on my shorlist but it's too complex for me (from what I've read on TOP and elsewhere), and I was sure I'd botch my pick of a 2-3 lens outfit because of the embarrassment of native m4/3 lens choices. I finally settled for an ILC (partly on the say-so of TOP reviews and comments) with an APS-C sensor and middling ISO and resolution, nearing its past-due date which kept getting postponed because of timely firmware upgrades (which also keep its price from dropping!).

Ever since I read that Fuji X-Pro1 review and its follow-up (Photons? Who Needs Photons?!), I've been dreaming of shooting fireflies by moonlight and star trails.

Unfortunately, after only 3 months of use, my camera unit conked out (an undocumented electronic shutter failure) before I got around to shooting nightscapes. In fact, it died on me before I could try all of its features and "perfect" my technique. It took great pictures (according to me) in the daytime though, in- and outdoors.

While waiting for my "old" camera to get repaired or replaced under warranty (I sent it back twice already), I bought a fixed-lens, small-sensor "advanced" point-and-shoot from the same maker, as my "back-up" camera. I'm not emotionally invested in the brand (although moneywise, the old camera took a big chunk from my kitty). I bought it because it has the same UI as its big-brother and even more stills shooting features (which will improve with the latest FW upgrade which I haven't installed yet). This fourth-generation, top-of-the-line p&s also has a composite time-lapse shooting workaround for capturing star trails!

I've taken a few moon shoots with it and I'm liking it. Next on the agenda is star trails (and hopefully fireflies), during my up-coming provincial trip far away from Manila's light pollution. Because of its small sensor, I expect some noise. But it shouldn't be much noisier than the Fuji X-Pro1 night photos which came with the follow-up review.

When I get my old camera back in working order and after more testing of its features, I'm going to make a side-by-side "user review" of these siblings (paying heed to this column and the comments re reviewer bias). As an enthusiast, this (user review) is all I'm qualified to do. I think I have the "constitution" to make a critical review, and intimate familiarity with a camera ought to count for something. That sounds like a plan and I'm feeling better already.

Thanks, Ctein!


It is because of these difficulties that affect the tone of published reviews, that I particularly like to know what the reviewers have bought and use in their personal kit -- and to know that they paid a price that is available to the general public (no kickbacks).

The review can say this or that, but if the reviewer liked it so much he just had to go out and buy one after returning the loan unit, that impresses me.

I strongly agree with Joe's featured comment. In the 1980s one of the film reviewers at the Minneapolis Star Tribune was hugely valuable to me (I don't remember his name). We were almost diametrically opposed in our tastes, but that still helped. I knew that if he hated a movie, there was a better than average chance I'd find it amusing, especially if it featured lots of explosions. Mindless escapism for the win! And if he loved a movie, I'd almost certainly find it either too "arty" or too depressing for my tastes.

There are at least two major kinds of "review" out there.

The first is what Ctein is talking about, where the purpose is to inform potential customers about a thing, so that the customer can evaluate the suitability of the thing for their needs (customer in quite a broad sense, here). This type of review is NOT aimed particularly at the reviewee, the point is not to drive positive change in the thing being reviewed. Ferreting out the negatives isn't really the purpose here, although it's a element. The goal is to try to create a holistic and fairly complete idea of the thing under review. As has been remarked, in a mature industry we don't really expect a lot of marked negatives or positives. The products tend to be pretty much as described on the tin. The point is to expand a bit on what it says on the tin and try to address what customers will or will not appreciate about the thing.

The other type of review is targeted to the reviewee, and the purpose is to drive change in the thig being reviewed. Employee evaluations, code reviews, internet critique of amateur photographs, and so on. It's absolutely vital to hit the negatives in these things, since that's the point -- ferret out the negative so it can be removed. Ferreting out the positive specifically isn't actually that important technically, but it can be vital psychologically.

Well there are reviews and then there are reviews. I am not sure that the reviews that one sees on, say, Amazon for a particular product are in the same category as the reviews that one reads on a website where the business model is to attract eyeballs with the review. With Amazon's product reviews, I am often looking a) at the number of reviews, b) the ratio of good reviews (4's and 5's)to bad reviews, b) for trends in the bad reviews (e.g. poor customer service, weakness of a particular part, interface problems). This is particularly true when shopping for something in a product category I know little or nothing about.

Then there are reviews by tyros who are letting the rest of us know about their own successes in finding the tools they need. Like Mike's recent DAC recommendation. Or even for a CD player (gasp). How in the world is an uninitiated consumer supposed to decode all the marketing blather out there?

I do think though that the reader has an obligation to bring his own bias to the review, if you know what I mean. For example, you have got to know how you feel about Pauline Kael to know whether to place any credence in her movie reviews. Insert your own reviewer and subject matter here, if you know what I mean. Personally, I like Michael Reichman's reviews because most of the stuff he looks at is stuff I would only purchase if I won the lottery. Hey, once we are in that territory, it's all entertainment for me.

Ben

I've been thinking that any article which discuss a camera's features but makes no mention of limitations or faults is not a review but an advertisement.

Sure we need announcements to inform us of new gear, but we also need to know what its weaknesses are, even more than what it's good at.

Well, I have a huge problem with all of this for 2 reasons 1. The conflation of "reviewing" and "criticism". They are absolutely not the same thing, and can be diametrically opposed. 2. My biggest problem, though, and I'll be critical here, is that the general, common (as in "low") usage of the word criticism always has a negative connotation,and this is a very puny way to understand the word. Instead, think literary criticism (and not book reviews!!!) and get a better understanding of what criticism is all about at its "higher" levels. It is not about running down a work or finding flaws in it, but a tool to open up the work for deeper understanding. That's a long way from a review. It's an exegetical exercise, and it is not fixed but evolves over time as new contemporary contexts are brought to bear on the work.

It just drives me nuts when I see/hear the concept of criticism so circumscribed by a false limitation.

One of my guilty pleasures is reading bad reviews. A well-written review of something truly awful is a delight. (A.O. Scott, the movie reviewer for the NY Times is a star in this sub-genre.)

Another guilty pleasure is reading TOP when I'm supposed to be working. Bye.

Dear Sarge,

I'm glad you enjoyed the Fuji piece, but as I said in the second paragraph, “I'm going to get three columns out of my experiences with it...none of which will be a review of the camera."

It was not a review, not by any stretch. My first impressions of the camera were poor enough that I knew I wasn't going to be interested in it so I didn't bother to give it a full test. It didn't make it past the first cut, in other words.

Now, if someone had been PAYING me decent money to test it, that would be a different matter. I'd grit my teeth and put it through its paces. But on my own time and (mostly) on my own buck? Wasn't going to waste that.

Which does get to my point in the column about reviewers being disinclined to test equipment they don't like.

A proper review that was critical was my review of the Olympus 12 mm f/2 lens ( http://theonlinephotographer.typepad.com/the_online_photographer/2011/11/olympus-12mm-review.html ). Which the fanboys hated, by the way.


pax \ Ctein
[ Please excuse any word-salad. MacSpeech in training! ]
======================================
-- Ctein's Online Gallery http://ctein.com 
-- Digital Restorations http://photo-repair.com 
======================================

Dear arg,

You're going to have to disallow and discredit almost everybody in the business based on your criteria, and that includes me. The standard practice in the industry has been that reviewers have the option to purchase equipment they have reviewed at dealer price (averages about a 25% discount off of list, although it can vary anywhere between 5% and 40%) if they like it enough to want it. Also, anything under $100 doesn't get returned, ever; the manufacturers don't want it back.

(We're talking about physical goods, here. Software, having close to zero delivery cost and no resale value for the manufacturer, never gets returned.)

There are only two classes of reviewers who don't get discounts and freebies like this. Ones who are so little known in the industry that they have no connections and no credibility and a very, very few, like Michael Reichmann or Lloyd Chambers, who are sufficiently wealthy that they can afford to buy anything they want to play with, off the rack. (Come to think of it, I don't even know that they don't get the discounts. Never asked. ) I think there are a couple of reviewers out there who proudly pronounce that they never accept stuff from manufacturers, they always just buy it out of pocket. That doesn't make them especially good at reviews.

Getting the discount has no effect whatsoever on the quality of the reviews. You're not going to find many reviewers who will give a favorable review to a piece of equipment they don't like so they can buy it at 30% off! (I've known exactly one person in the business in 30 years who “churned” equipment for profit, and everyone knew he was a bad egg. There's always someone unscrupulous who can game any system)

You are right that it is generally very high praise when a reviewer says that they decided to buy the piece of equipment they were testing. I've hardly ever done that, and I make a point of saying if I have, because it really is high praise, 30% discount or not.

Also, buying equipment to review introduces its own set of psychological quirks; one becomes invested, both positively and negatively, in what one has spent one's own cash on. So the buying reviewers can't guarantee you a more “objective” point of view.

That's the thing: a good reviewer isn't immune to the pitfalls of human psychology around possessions. They're just better at recognizing and dealing with them.


pax \ Ctein
[ Please excuse any word-salad. MacSpeech in training! ]
======================================
-- Ctein's Online Gallery http://ctein.com 
-- Digital Restorations http://photo-repair.com 
=====================================

Dear Tex,

I think you're reading into this things that aren't there. I don't notice anyone conflating reviewing and criticism here-- not me, nor any of the commenters. In fact Criticism, as an activity, is only mentioned in passing.

Indeed, many people in the world can't tell the difference between the two, and as an English major that vexes me also. But I don't see the sin being committed by our readers.

pax / Ctein

My biggest problem, though, and I'll be critical here, is that the general, common (as in "low") usage of the word criticism always has a negative connotation, and this is a very puny way to understand the word. Instead, think literary criticism (and not book reviews!!!) and get a better understanding of what criticism is all about at its "higher" levels.

Criticism as in "literary criticism" is only for literature and other art forms, not equipment. Books, art—including photography, architecture, and movies can be the subject of "higher" criticism. One can even deconstruct TOP articles (and comments) as text, if not literature.

Qualifying equipment reviews as "critical" is not conflating reviewing with criticism. As this column and many readers point out: "there are reviews and there are reviews". There's a need to distinguish an uncritical "review" from the real thing, is all.

The common ("low") usage of criticism (of which there has been a slew of articles and comments in TOP lately) didn't read to me as "criticism" in the literary ("higher") sense. Even in common usage, there's such a thing as constructive, uncynical criticism.

With respect to camera and lens reviews, I think the "higher" equipment review form is what Roger Cicala at lensrentals.com does: equipment teardowns. Now, that's literal deconstruction.

Here's my layman's take on "higher" criticism (deconstruction).

Sarge - I see what you're saying, but Ctein did mention "reviewers and critics" in his post. Twice I think

Dear Sarge,

Thank you for the defense, but Richard is right. I did refer to both reviewers and critics. What I didn't do is conflate them. It's really no different than if Mike were to write one of his automotive columns and referred to “cars and trucks” or another audio column and referred to “loud speakers and headphones.” It doesn't mean he's confusing or conflating the two; he's just referring to both. That was simply a minor error in interpretation on Tex's part, and I'm in agreement with him–– people shouldn't conflate them.End of argument, no harm no foul.

It does hark back, going full circle, to Richard's original observation, which is that the word criticism has gotten a bad rap because it's most commonly used to mean, well, being critical (as in negative). That's a problem with language that we're just stuck with. Similar to Richard's peeve, mine is people who insist on telling you what you meant by using a definition of the word different than the one you did. It can be an understandable misunderstanding, but when you tell them that, no, you meant B rather than A, they insist you're wrong. Even more annoying is when they try to argue that you're wrong because A is the "more common” or the “first entry in the dictionary” meaning.

To which my mental, rarely spoken thought is, “and what part of the phrase, 'more than one' don't you understand?!” If words were meant to be used with only one meaning, there would only be one meaning.

Us English majors can get awfully cranky at times [VBG].

But getting back to your point, since I think we've exhausted my original column topic, I think of there really being three types of endeavor rather than two. There is being a reviewer, there is being a critic, and then there's the activity called criticism, as in literary criticism. Which I think of as somewhat distinct from the second, it's really more of an analytic procedure. Of course one blends into the other, but to me these feel like three different activities. I'm a good reviewer. I'm also good at criticism. I'm a pretty lousy critic.

pax \ Ctein
[ Please excuse any word-salad. MacSpeech in training! ]
======================================
-- Ctein's Online Gallery http://ctein.com 
-- Digital Restorations http://photo-repair.com 
======================================

The comments to this entry are closed.