Editorial Note

Ctein has had to reverse his assessment of the Electra MAX 3G 960GB SSD drive he wrote about on Wednesday. Please see his addendum at the bottom of his post for details.

—Mike the Ed.

Comments

[adamct]

Yesterday, I decided to buy an SSD to replace my system drive. As usual, the first place I went for buying advice was tomshardware.com, where they have an article called "Best SSDs for the Money". It is broken up into various budget tiers. My budget was about $200, which worked out well, because I wanted a 256GB drive. There are several drives around the $200 mark, but I noticed this comment in the roundup:

"We regularly have readers point out that they aren't necessarily concerned about the highest performance, particularly since many of the numbers we measure aren't commonly seen during day-to-day use. They're more worried about long-term dependability."

This comment was made in the context of a recommendation for a Crucial m4 drive, which had slightly lower performance numbers than the competition, but was deemed to be highly reliable. Which drive did I buy? The m4.

This is NOT intended as a criticism of Ctein, nor is it supposed to show how brilliant I am, I'm just happy to see that tech reviewers are beginning to focus less on pure performance, and are taking reliability into greater consideration.

Best regards,
Adam

[ctein]

Dear Adam,

No problem; I didn't take it as a criticism at all. This is just what a reviewer's life is like, Murphy's Law and all.

By the way, I've posted a detailed comment on the situation to the comments on my review of Wednesday.

Now, as for the prospects for reviewers taking reliability into consideration, this is almost impossible. (I think I may have written a column about this, even, but I can't remember. If I have, with some reader please point to it? Thanks!)

Most consumer products have a several percent failure rate out-of-the-box. It's just a random occurrence, and a reviewer who reports that to the readers without a deeper underlying reason isn't really doing anyone a service. It's like reporting that because one roll of the dice came up two that proves you're fundamentally an unlucky person.

Over my reviewing career, I would've had to damn many good products, just based on those percentages.

Worse, there is a plausible chance that I could get TWO out-of-the-box defective products in a row once in my career. Understand, I'm not talking about products that are more inherently defective, I'm just talking about how the dice roll.

Consequently, it's almost impossible to write about reliability without a very deep technical understanding of what you're dealing with. Like the columns Norm Rothschild (and to a lesser extent Arthur Kramer) used to write for Pop Photography. Or, you have to have access to massively crowd-sourced information, like Consumer Reports has.

I don't know anything about how tomshardware gets their reliability information. Without knowing exactly how it's arrived at, I have to consider it meaningless. Mind you, Crucial has a good reputation… I'm not saying you made a bad choice. But the data lacks intrinsic value.

To give you a concrete example, I've had such a run of bad drives from Seagate that I damned them in a previous column. I'll still stand by that, but several readers whose lives have required them to deal with hundreds of hard drives quickly reported that they had seen no unusually high failure rate with Seagate drives. I'll still avoid buying one myself, because I'm human and subject to prejudice, but that's about me.

Other things that confound reliability is that it's not consistent from product to product or even over the lifetime of a product. The Pentax 67 was considered a veritable hockey puck of the camera… except for a handful of years in the early-middle of its production run where Pentax made some kind of internal gearing change that made them relatively unreliable. And then they changed back.

The bottom line is that almost all reliability information you read is hearsay and if it comforts you to read it, then well and good. But it's not telling you very much you can take to the bank.

pax \ Ctein
[ Please excuse any word-salad. MacSpeech in training! ]
======================================
-- Ctein's Online Gallery http://ctein.com 
-- Digital Restorations http://photo-repair.com 
======================================

[Pedro Côrte-Real]

>I think I may have written a column about this, even, but I can't remember. If I have, with some reader please point to it? Thanks!

Not exactly reliability but could you be thinking of your Epson 3880 columns? The ones that dealt with manufacturing variability.

[adamct]

Ctein,

All good points, but I think you misunderstood what I meant when I wrote that my comment wasn't intended as a criticism of you. Lest that sentence itself be misinterpreted, let me explain. I meant that my comment wasn't intended as a criticism of you as a BUYER/CONSUMER (that's why I also added that my little story doesn't prove I'm a brilliant shopper). And my comment about being glad that tech reviewers are taking reliability into consideration was an independent statement and was merely supposed to be a compliment of Toms Hardware. I can see how you would read them as related comments, but they weren't. It frankly wouldn't even occur to me to criticize you as a REVIEWER (at least not over something like this).

For what it's worth, I took the Tom's Hardware comment on the m4's reliability as based on (i) their personal experience using it in their lab for 18 months, and (ii) it's widespread adoption in the marketplace with few reported problems. Neither is scientific or reliable. But in the absence of better information, it's all I have to go on.

Best regards,
Adam

[Robert Roaldi]

In a previous life, I wrote firmware for a living. We'd discover the oddest bugs all the time, wtf, how did that get past testing? Our favourite saying became that it was a miracle from god that our cars started every morning.

[ctein]

Dear Robert,

Heh. I got to feel the same way about photography and printing. The more ways I found that things could go awry in the darkroom, the more I marveled that anyone (moi included) ever managed to make a decent print.

People who think they all have it under control have not appreciated the gravity of their situation. [s]

pax / Ctein

[Richard Tugwell]

re Robert Roaldi - spotted this (I'm paraphrasing) on twitter recently about the stages of debugging

1 But that's impossible!
2 How on earth could that happen?
3 Wtf???
4 How on earth did that ever work?