Relationships

As a further comment to Ctein's "Payola," below, another fact of life in product reviewing is that it's virtually impossible for any magazine or website to establish and maintain the same relationship with all companies equally.

That's true when some of the companies advertise with you and some don't, but it's also true even in the absence of the advertising influence. Even the biggest publications have some sort of natural bias; their staff has certain interests, and uses and likes certain types of products. Naturally, they'll tend to attract a readership that is also interested in those things. Then, to get to that same readership, the companies that make appropriate products gravitate toward that publication.

For many years, for example, Photo Techniques was edited by one practicing sensitometrist (David Alan Jay) and had another as a Contributing Editor (and de facto technical editor, although he never held that title)—Phil Davis, author of Beyond the Zone System. Phil wrote many articles centering around sensitometry, and David was supportive of that type of content.

In fact, when I took over from David Alan Jay, he said to me that to "fully participate" in his magazine, a reader needed to own three things: a sheet-film camera, a spotmeter, and a densitometer.

It just happens that some companies will like and support certain publications, and vice-versa, and a relationship builds. Certain relationships grow and flourish while others wither and go moribund. As time goes on, that process tends to reinforce itself. Remember that many companies have very small advertising budgets. They might be able to place their ads in only a handful of publications, or only two, or just one. They strategize very carefully to try to figure out where an ad placement will do them the most good. Certainly, X-Rite wasn't going to try to have its latest densitometer reviewed in Petersen's Photographic, and Kodak wasn't going to advertise disposables with us.

After some years have gone past, you find you've known certain business associates for a long time. They might even become friends. Sometimes advertising is indeed a part of that mix; sometimes it isn't. The relationships tend to ebb and flow anyway. That seems both natural and unavoidable.

____________________

Mike

Featured Comment by John Camp: "This topic, perhaps cast in somewhat wider terms, is one that whole journalism courses are devoted to, and it's not an easy one. The most recent really big blow-up on the camera-oriented 'net, as far as I know, was Michael Reichmann's predicament with his review of the Leica M8 on his Luminous Landscape.

"(I won't explain here, but if you google 'Reichmann Leica M8 magenta review,' you'll find more than you probably want to read.)

"A lot of people have reflexive reactions to this kind of thing (usually, publish and damn the consequences), but the longer you think about it, the more troubling the consequences become. In my view, Reichmann was more-or-less screwed the moment the Leica turned up in the mail. At that point, something bad was going to happen no matter what he did.

"Interestingly enough, though, he then published a column which recounted his reactions to the M8, and his decision not to refer to certain problems with the camera. That's when the stuff hit the fan—but that's exactly at the point in which I decided that he was more trustworthy than I otherwise might have expected. He could have kept his mouth shut, and no one would have been the wiser; instead, he admitted what he'd done, and explained why.

"This whole issue is a bottomless bucket. IMHO, there is no 'answer' to the problems created; it's something that has to be dealt with on a case-by-case basis. In the Reichmann case, I believe I might have done the same thing he did, right down the line. I just wonder if I'd have had the balls to admit it later."

Featured Comment by Ken Sky: "Start with the premise that no one can be totally without bias. An honest reviewer lays out his/her bias at the beginning of the review. The review should provide as much objective information as possible (data) and when the reviewer draws conclusions all caveats should be pointed out as well as the criteria for these conclusions. Unfortunately, we as consumers/audience are also part of the distortion. How many of us can honestly say that we have read the article the first (only) time from beginning to end before skipping to the conclusions? If we do the latter we become biased as to how we absorb the data. That's what leads to all the fighting on internet fora."

Mike responds: Ken Sky raises a point that's vivid to writers but seldom acknowledged on the internet: readers often read poorly, incompletely, or with their whatever-colored glasses on. Sometimes they reach closure early, occasionally way too early. I'm aware of one phenomenon I call the "imported argument": it's when we raise the specter of an issue that a commenter has long been arguing on another forum, and he brings to our article all the arguments he'd developed on that other forum—sometimes without bothering to comprehend the subtleties of the argument being made here. I'm good at recognizing this.

Then, just as surely, we all tend to read for affirmation: a person who badly wants to think that a camera is great, for example, might gloss over or miss the negative points stated right there in black and white under his nose. But of course the most common error is simply quick scanning and sloppy reading. I'll often receive questions that were explicitly answered within the article. Just as listening is a skill that requires intelligence and attentiveness and that can be mastered, so is reading. Of course, good and bad readers might be the same person—each of us—in different situations or on different days. Habitual careful reading is relatively rare, I would guess.

Comments

[michael]

Years ago, "The Wall Street Journal" called "Car and Driver," 'a chrome plated conflict of interest.'

The same could be said of the "Journal."

Few writers are going to antagonize their best advertisers. I'm not being cynical; it's just a fact of life.

That's why "Consumer Reports" does not accept advertising, or allow their reviews to be used by companies. Before I buy something, I read PopPhoto, dpreview, LL and anything else - and draw my own conclusions. I look at photographs taken with a camera/lens, and take it from there. I don't bother with digicams. There's only so much a tiny sensor can do, especially with so many diodes packed on that tiny real estate. The only decent digicam I ever found was the Fuji F10. But now they've gone pixel crazy, so I guard my F10, because I'll have trouble replacing it. Pany did something I love: they went to 7.5 MP for the L1, instead of packing the chip. The photos are superb, and the resolution is the same as 8MP.

Your thoughts?

[Cateto/Jose]

Hi Mike,

Not sure if this is a somehow direct reply to my comment yesterday to Ctein's Payola article (http://theonlinephotographer.typepad.com/the_online_photographer/2008/07/payola.html#comment-123403546).

I completely understand your point of view, and fully agree: reviewing photography material is a human activity, and thus we can't (and shouldn't) avoid the effect of human relationships. Absolute objectivity in a review, as posted by others, is just impossible. And rather than objectivity, I think we should look after a fair review of products: that is more than enough in my book.

But having said that, let me say that my post yesterday addressed a couple of very ugly examples (and without citing names, I guess most readers of TOP know the magazines, websites and dSLR companies I addressed). In those cases, the situation goes far beyond the normal and get into plain reprehensible behaviour, as they are not honest: in both examples, what happens (IMO) is paying good opinions with money: that clear.

In my first example, a certain company was openly utilizing their advertising budget in order to pressure the editorial content in a high profile magazine. Given the opinion was not good on the flagship dSLR model of that company, they stopped advertising. Isn't that ugly enough?

In my second example, a certain website has been waiting for many months to review the flagship models of two dSLR manufacturers that are not among the top two. Yet every time the top dSLR company releases a new dSLR (be it high, average or low level), you can bet a preview will be posted before the actual availability of the model and then, a fully fledged review is usually posted around the public release of such model. Pretty much every connoisseur in this world knows the reasons for that. And pretty much every connoisseur in this world knows that, ahem, the top dSLR maker will receive a much different response in this particular website, when compared to other manufacturers.

Both examples illustrates how things should NOT be done, and go far, far beyond the acceptable.

So to conclude: relationships matter, sure, and different places (magazines, websites) have different relationships, thus their editorial opinion is different. Sure, and fine: diversity is the key of life.

Now diversity is one thing, and selling the soul to a company to get big bucks in advertising is another.

Let me finish by stating that my posts are not, in any way, a call against advertising: please don't misunderstand me. I know that advertising is useful, and needed in order to ensure the existence of private magazines and websites. What I say is a call against the bad use of advertising budgets as hidden ways to push the opinion of high profile magazines/websites.

And I would also like to say, clearly, that I appreciate a lot the independence of websites like The Online Photographer, where you can see all kinds of opinions about all kinds of manufacturers, and who is not afraid of posting controversial opinions even if they go against the "politically right" opinion in the world of photography. I remember, for instance, your recent TOP ten list of recommended cameras, which in my opinion is a perfect example of independence, even if more than one guy opened widely his eyes after reading, for instance, that the K20D has better image quality than the D300, although the later is in your opinion an overall better camera. I would like to read such honest (even if they are controversial) opinions in other places too.

[Roger Botting]

I have been sceptical of camera reveiws since the '70s when, I beleive, Pop Photo discussed their testing methods. Lots of good stuff but a couple of points that made me wonder. First, they would not publish reveiws of equipment that would not meet their standards. If it wasn't published, was it any good? or, what was wrong with it. One could also wonder about less savory aspects.
Second, they mentioned that on several ocassions that they would repair the evaluation cameras if they didn't work properly. As a buyer I would expect my camera to work properly from day 1. I think a reviewer should comment that if new equipment was defective on delivery, or the test equipment needed multiple repairs for the test to be concluded.
A review needs to be done by qualified reviewers, personal biases left aside, no commercial concerns, just a straight up honest reveiw.

[Jason Anderson]

This is exactly why people like Scott Kelby and the NAPP staff talk about Photoshop and not products like Aperture, or DPP. Occasionally they have mentioned the Nikon software but that is only because they all are Nikon shooters. Relationships develop by the sheer bias of interest.

This is a natural and inherent bias but people often will make a negative association with the term bias, since it connotes a subjective objective opinion or school of thought.

We all view the world through subjective and opinionated eyes, whether we want to admit to it or not. I think when it comes to product reviews, vendors will naturally trend toward those that they know have an affinity for their product line, and there's nothing wrong with that. The rest of us, as consumers, certainly know and expect a certain amount of bias in what we read.

This is why it's important to read across schools of thought. For example, I read WIndows IT Pro, but at the same time I also read MacWorld. I write a blog that is clearly subjectively titled as a "Canon Blogger", but this doesn't mean I can't appreciate the quality of Nikon, Olympus, or any other vendors line of products.

Asking someone to make a "straight up honest review" with "no bias" and you will find a very basic, and probably uninformative review. Find reviews that talk a product up (or down) and then look at the reviews of the alternatives. Whichever has the most pros and the fewest cons - there's your best option. Finding an unbiased review just isn't going to happen. We have a responsibility as consumers to recognize inherent bias and just accept that. Would you look for conservative schools of thought in the New York Times? Of course not! Likewise, liberal schools of thought are not too common in the Wall Street Journal. Neither bias makes either one a poor publication - it's just their bias. Recognize it, read it accordingly, and develop your own perspective based on which set of information aligns most with what your proclivities are.

[Yanchik]

I used to have a boss of whom I frequently had cause to say that he was kind enough to make his disadvantages and weaknesses immediately apparent rather than to postpone our despair by disappointing us later. I'm sure we can all recognise such people from our own experience...

It's probably good to try and see reviews in this sort of light. Treat them for what they're good. Bias is unavoidable; but without being complacent about it, is it really so bad ? After all, surely photography is the most extreme example of an activity in which we are required to form and test our own views ?

Y

[Ctein]

Jason: Ummm, no, not in common use. A bias is most commonly defined as a partiality that prevents objective consideration of an issue or situation. The word does not usually mean a mere preference or leaning.

If someone were to state that reviews of mine were biased I would consider myself insulted. It would also be untrue.

I could very well write a review that was badly in error (I'm human, and not infallible), but it would not be due to bias.

Cateto: I have no idea which websites or magazines you're talking about. Really. Especially since the magazine is in French. C'mon, give us names.

Michael: That's not a 'fact of life;' that doesn't reflect reality. I have no idea what happen in the C&D world (as I said in my column the different genres do NOT play by the same rules) but in the US photo world, the major advertiser was, of course, Kodak. Up until the mid-90's when they cut way back, they were always good for a back cover (expensive!) and some internal pages in just about every pub of any substance.

They were total mensches when it came to the press. I and many other reviewers wrote critically at times about both products and policy. They never suggested there would be any retaliation; they often stated the opposite.

That's just one of many coprorate examples.

(But many people believe as you do, and some of them do act upon it. I *have* known mags to be erroneously terrified of losing Kodak ads.)

Roger: How reviewers deal with broken equipment is, indeed, a complicated issue, and I ain't saying you're wrong (or right). But... are you basing your decisions on a policy statement from 30 years ago?? That's two (three?) full turnovers in staff and the same in management/ownership.

pax / Ctein

[Cateto/Jose]

Ctein,

I will be a nice guy and pretend I believe you when you say you have no idea on what magazine and website I am talking about.

As for the magazine, here is a short summary of the story and a direct link to the editorial of that magazine regarding this issue
http://heim.ifi.uio.no/~gisle/blog/?p=93

As for the website, sorry but I can't even pretend to believe you can't get it. As a hint, I can only tell you that it has the words "digital" and "review" in its title, and it deals with photography. Do you get it now?

;-)

[Ctein]

Dear Cateto,

I'm really glad you're a nice guy!

Honestly, I didn't know which magazine you meant (I have *NO* familiarity with the French pubs), and I could think of three or four possible websites.

I appreciate the enlightenment.

pax / Ctein

[Paul]

Well said, but I think there are also extra factors that come into play. The "Personality' where reviewers leave one title and go to another and expect their readers to do the same; 'Lifestyle' when reviewers compare products to other prestige items they own (imagine if a review ran: I would recommend this tripod because it fits comfortably in the boot of my Jag.); and of course 'The Snob' - I only own/use the best.
Sadly, those types of reviews or reviewers I think tend to push people away from the hobby, rather than bring them in. Mostly I believe by making newcomers nervous about the cost of getting into the hobby.
Just my two cents worth, but maybe you can write a satire about reviewing personalities... you know they are out there and writing about photography, audio, computers....

[Jason Anderson]

An excellent point Ctein, and I should probably have used the word "subjective" or "subjectivity" in place of "bias". Ergo, there is nothing wrong with a subjective review, as we are all somewhat subjective. Complete objectivity would be a good thing to aspire to, but I guess what I am saying is that if things like reviews and such were the result of completely objective writing, they would be so necessarily bland, that they would not be useful. I'd rather have someone write a review of Canon or Nikon who knows the product inside and out. Their review will thus be necessarily subjective since the probably found things they both like and dislike, and the subjective nature of human beings means we are precluded to see things in a particular light, whether we want to admit it or not. We can be objective, or act without bias, but we cannot help our own preferences, and that subjectivity will ultimately manifest itself in how we express ourselves.

Back on topic though - I can see how Michael's subject matter for today (relationships) would lead toward the development of relationships between reviewers and product vendors.

[John Camp]

Michael said: "Years ago, "The Wall Street Journal" called "Car and Driver," 'a chrome plated conflict of interest.' The same could be said of the Journal."

Of all the national papers, I find the Journal to be the generally most trustworthy, even though I am of the liberal persuasion. The reason for that is, I can identify the Journal's bias -- they are for business and money and making it, and against government intervention and taxes. The problem with the New York Times is that I can never quite figure out where they're coming from, and they are definitely coming from somewhere -- sometimes it's the liberalism that I generally approve of, but other times it's a kind of crazy populism, or its some kind of tactical Washington-insider deal. When a Journal columnist lies, you can usually figure out why, because the Journal generally reflects a coherent ideology. When a Times columnist lies, you can tell that he's lying, you just don't know why.

JC

[Robin P]

As one who nearly always buys a copy of Chasseur d’Images on my trips to France I can thoroughly recommend it as a photo magazine with its own style and authority.
Ctein, if you have a friend visiting Europe ask them to bring you a couple back (oh & BTW - they seem very predisposed to Pentax).

Cheers, Robin

[Ctein]

Dear John,

Because it is such a huge can of worms, one of my very few inviolate policies is that I will not review beta product (whether it's hardware, software, or media). Such products invariably have bugs (or they wouldn't be beta) and that puts me in the position of having to reverse-engineer the nature of the bug and figure out if it's going to be present (or at least of importance) in a production unit.

Absent a crystal ball and even more m4d l33t tek sk111z than I already have, I am going to make serious errors in judgment trying to do that. The cost of that was well demonstrated by poor Michael. He incorrectly evaluated the seriousness of a problem and that led to all hell breaking loose.

As a consumer, I love quick-appearing reviews. I just hate it when I'm thinking about buying something and can't find a good review of it. Readers understandably want instant information, but fortunately I don't have to listen to their demands. This is one of those cases where the reader either gets it fast or gets it good, and as the writer, I'm not willing to give them other than good.

~ pax \ Ctein
[ please excuse any word salad. MacSpeech in training! ]
======================================
-- Ctein's online Gallery http://ctein.com
-- Digital restorations http://photo-repair.com
======================================

[Cateto/Jose]

Dear Ctein,

My comments above were lighthearted and intended to be a joke.

But now I believe you didn't know about the magazine. And I agree, although I addressed a specific website, similar stories could probably told about a few more.

That's why I always advise friends that you have to be very careful with the stuff you read on the internet, as there are some jewels, and too many places that don't deserve to be trusted.